Kubernetes cannot pull image from private docker image repository

Question

I have problem with kubernetes (minikube) and pull images from local image repository on docker. Docker repository was created:

docker run --entrypoint htpasswd registry:2 -Bbn zordon examplePassword > /mnt/LINUX/auth/htpasswd

docker run -d \
  -p 5000:5000 \
  --restart=always \
  --name registry \
  -v /mnt/LINUX/dockerreg:/var/lib/registry \
  -v /mnt/LINUX/auth:/auth \
  -e "REGISTRY_AUTH=htpasswd" \
  -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
  registry:2

Then I want to create simple pod with image which was succesfully uploaded to local repository:

curl localhost:5000/v2/_catalog
{"repositories":["car/configuration"]}

I have also create secret on minikube cluster with:

kubectl create secret docker-registry docregkey --docker-server=localhost:5000 --docker-username=zordon --docker-password=examplePassword [email protected]

and define simple Pod:

    apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
  - name: private-reg-container
    image: car/configuration:latest
    imagePullPolicy: Always
  restartPolicy: Always
  imagePullSecrets:
  - name: docregkey

unfortunatelly I getting still:

Failed to pull image "car/configuration:latest": rpc error: code = Unknown desc = Error response from daemon: pull access denied for car/configuration, repository does not exist or may require 'docker login'

How i can fix this problem ?

Answer 1

For minikube to pull from your own local docker registry, the tag affects the pull policy. Per Images docs, pull policy is IfNotPresent by default EXCEPT if

1. you use :latest as the tag for the image to use
2. OR you omit the tag for the image to use.

In those cases the pull policy will effectively default to Always, which will attempt to pull from docker hub. This will cause minikube to be unable to fetch local images that have no tag or "latest" tag.

Moral of the story is, don't rely on the default because it is too confusing :)

So always explicitly state the pull policy:

1. when deploying into minikube the pull policy should be IfNotPresent or Never for the local images
2. when deploying into a cloud host (like AWS), pull policy should be as for public images (see below)
3. the pull policy should be Always for those public images that use a tag like "latest" or "stable" (because the image the tag points will change over time), and IfNotPresent for tags that always point to the same image (to avoid fetching more than necessary)

This means that if you avoid using tags like latest and stable etc, there is only one rule to follow:

1. explicitly set the imagePullPolicy in your spec (or on the command line in the case of run) to IfNotPresent, as this is will always look for it locally first, and go to public registry if it is not found locally, and this will work whether or not you are deploying into minikube or cloud.

Answer 2

The problem is with the image pull policy - you have set this to Always (the default setting). This means that the Docker deamon always tries to pull the image from the outer Docker registry - you want to use the local one instead.

Try to add --image-pull-policy=Never when creating a deployment.

A good tutorial of using locally created images is here (it helped me):

https://kubernetes.io/docs/tutorials/hello-minikube/#create-a-docker-container-image

Answer 3

Because Minikube is VM not a your localhost. You try this code eval $(minikube docker-env) https://kubernetes.io/docs/getting-started-guides/minikube/

1. Open Terminal
2. eval $(minikube docker-env)
3. docker build .
4. kubectl create -f deployment.yaml

just valid this terminal. if closed terminal again open terminal and write eval $(minikube docker-env)

eval $(minikube docker-env) this code build image in Minikube

Answer 4

I wanted a one line solution to execute in my terminal. Everything else I tried was overly complex to auth ecr with minikube.

This is my command for aws ecr login that I run each day because the token expires. The examples below are for Debian 9 with AWS ECR.

shell

kubectl create secret docker-registry aws-ecr-credentials \
--docker-server=$ECR_REGISTRY \
--docker-username=AWS \
--docker-password=$(aws ecr get-login | awk '{print $6}') \
--docker-email=$IAM_EMAIL \
--namespace=$KUBE_NAMESPACE

template.yml

spec:
  imagePullSecrets:
    - name: aws-ecr-credentials

Answer 5

The Problem is with the image name you are mentioning in the POD yaml file.

image: car/configuration:latest

This will try to pull from the global registry rather than local registry.Change the image name to include the repository too.

image: localhost:5000/car/configuration:latest

And make sure that you have included insecure registry in your docker daemon configuration if your registry is not secured.

Answer 6

When you run Kubernetes in Docker for Desktop your applications will share the same image registry across Docker and Kubernetes. List od all images:

docker images --all

Choose of them and run it with changed atribute --image-pull-policy=Never. For example:

kubectl run ContainerName --image=myimage/server --port=8080 --image-pull-policy=Never 

By default, the kubelet will try to pull each image from the specified registry. However, if the imagePullPolicy property of the container is set to IfNotPresent or Never, then a local image is used (preferentially or exclusively, respectively). Link

It's mean, that Kubernetes pull image from local registry, not remote cloud.

Answer 7

Private registry in Minikube

kubectl create -f kube-registry.yaml

(Grab kube-registry.yaml from this gist on github.)

and you need port-forward minikube to localhost (Just image build time)

kubectl port-forward --namespace kube-system \
$(kubectl get po -n kube-system | grep kube-registry-v0 | \
awk '{print $1;}') 5000:5000

After this, from the host curl localhost:5000 should return a valid response from the docker registry running on minikube

Repo : http://localhost:5000/v2/_catalog

Pull image : localhost:5000/image_name:image_tag

Reference: https://blog.hasura.io/sharing-a-local-registry-for-minikube-37c7240d0615

Answer 8

The same issue with me was that when I build an image with the local Docker which then is not added to minikube image list.

The resolution was to add the image manually:

minikube image load image-name:tag