GKE pulling images from a private repository inside GCP

Question

I've set up a private container registry that it is integrated with bitbucket successfully. However, I am not able to pull the images from my GKE Cluster.

I created a service account with the role "Project Viewer", and a json key for this account. Then I created the secret in the cluster/namespace running

kubectl create secret docker-registry gcr-json-key \
 --docker-server=gcr.io \
 --docker-username=_json_key \
 --docker-password="$(cat ~/code/bitbucket/miappsrl/miappnodeapi/secrets/registry/miapp-staging-e94050365be1.json)" \
 [email protected]

And in the deployment file I added

...
      imagePullSecrets:
    - name: gcr-json-key
...

But when I apply the deployment I get

 ImagePullBackOff

And when I do a kubectl describe pod <pod_name> I see

Failed to pull image "gcr.io/miapp-staging/miappnodeapi": rpc error: code = Unknown desc = Error response from daemon: Get https://gcr.io/v2/: dial tcp: lookup gcr.io on 169.254.169.254:53: no such host

I can't realize what I am missing, I understand it can resolve the dns inside the cluster, but not sure what I should add

It's a private cluster? Do you have network policies? or any security configuration? — David C
@DavidC Thanks for your comment, I think that maybe I am missing to set up a network in the cluster. — agusgambina
Could you share the Cluster network settings? like if it's a private cluster or if you set network policies or VPC Service Control? Check the next information about Setting up Container Registry or Artifact Registry for GKE private clusters maybe it's the case. — David C
@DavidC I think you were right, the problem was that it was a private cluster. I removed, created a new one (Public) and I was able to pull the images. Next time I create a private cluster I will need to setup the network to work with container registry. Thanks — agusgambina
I will write an answer on your post (it's different than a comment) to help other users with the same issue, if the information help you to resolve your question, feel free to vote up the answer. — David C

David C David C · Accepted Answer · 2021-02-19T16:18:56

If a GKE Cluster is setup as private you need to setup the DNS to reach container Registry, from documentation:

To support GKE private clusters that use Container Registry or Artifact Registry inside a service perimeter, you first need to configure your DNS server so requests to registry addresses resolve to restricted.googleapis.com, the restricted VIP. You can do so using Cloud DNS private DNS zones.

Verify if you setup your cluster as private.

GKE pulling images from a private repository inside GCP

1 Answers