Failed to pull image - unauthorized: authentication required (ImagePullBackOff )

Question

My release pipeline runs successfully and creates a container in Azure Kubernetes, however when I view in azure Portal>Kubernetes service> Insights screen, it shows a failure.

It fails to pull the image from my private container repository with error message 'ImagePullBackOff'

I did a kubectl describe on the pod and got below error message:

Failed to pull image "myexampleacr.azurecr.io/myacr:13": [rpc error: code = Unknown desc = Error response from daemon: Get https://myexampleacr.azurecr.io/v2/myacr/manifests/53: unauthorized: authentication required.

Below is a brief background on my setup: I am using Kubernetes secret to access the containers in private container registry.

I generated the Kubernetes secret using clientId and password(secret) from the Service Principle that my DevOps team created. . The command used to generate kubernetes secret:

kubectl create secret docker-registry acr-auth --docker-server --docker-username --docker-password --docker-email

I then updated my deployment.yaml with imagePullSecrets: name:acr-auth

After this, I ran my deployment and release pipeline both ran successfully, but they show failure in the kubernetes service with error message 'ImagePullBackOff' error.

Any help will be much appreciated.

Answer 1

As the error shows it required authentication. As I see from your description, the possible reason is that your team does not assign the ACR role to the service principal that your team creates, or you use the wrong service principal. So you need to check two things:

1. If the service principal you use has the right permission of the ACR.
2. If the Kubernetes secret was created right in the Kubernetes service.

The way to check if the service principal has the right permission of the ACR is that pull an image in the ACR after you log in with the service principal in docker server. Also, as the comment said, you need to make sure the command is right as below:

kubectl create secret docker-registry acr-auth --docker-server myexampleacr.azurecr.io --docker-username clientId --docker-password password --docker-email yourEmail

Additional, there is a little possibility that you use the wrong image with tag. By the way, check it out.

Answer 2

It's odd, maybe it shows an old deployment which you didn't delete. It may also be these; incorrect credientials, acr may not be up, image name or tag is wrong. You can also go with aks-acr native authentication and never use a secret: https://docs.microsoft.com/en-gb/azure/container-registry/container-registry-auth-aks

Answer 3

In my case the problem was that my --docker-password had an special character and I was not escaping it using quotes (i.e. --docker-password 'myPwd$')

You can check your password is correct my executing this command: kubectl get secret < SECRET > -n < NAMESPACE> --output="jsonpath={.data..dockerconfigjson}" | base64 --decode

Reference: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/