Authorizing to Kubernetes on Google Container Engine without gcloud SDK from Jenkins

3
votes

I'm trying to setup a continuous integration job that will deploy to kubernetes / google container engine from a Jenkins job. The jenkins server is relatively tightly controlled, so I'm not able to install plugins.

I have a JSON key file for a server account from Google Cloud IAM.

I'm currently trying to download the google cloud sdk and auth from there, but am not having any luck (this if from a Jenkinsfile):

sh 'export KUBECONFIG=$(pwd)/.kubeconfig'
sh 'export GOOGLE_APPLICATION_CREDENTIALS=$JSON'
sh 'google-cloud-sdk/bin/gcloud auth activate-service-account --key-file=$JSON'
sh 'google-cloud-sdk/bin/gcloud config set core/project proj-1'
sh 'google-cloud-sdk/bin/gcloud container clusters list'
sh 'google-cloud-sdk/bin/gcloud container clusters get-credentials clust-1 --zone us-east1-c'
sh 'kubectl get pods'

I'm getting the error message: error: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information. I will also need to be able to do a gcloud docker push, so using gcloud is ok.

1
kubernetesgoogle-cloud-platformgoogle-kubernetes-engine

1 Answers

3
votes

There's a cryptic github issue around this behavior:

https://github.com/kubernetes/kubernetes/issues/30617

According to that issue, everything I've been doing should work:

Previously, gcloud would have configured kubectl to use the cluster's static client certificate to authenticate. Now, gcloud is configuring kubectl to use the service account's credentials.

Kubectl is just using the Application Default Credentials library, and it looks like this is part of the ADC flow for using a JSON-key service account. I'll see if there is a way that we could make this nicer.

If you don't want to add the export GOOGLE_APPLICATION_CREDENTIALS="/path/to/keyfile.json" to your flow, you can reenable the old way (use the client cert) by setting the cloudsdk container/use_client_certificate property to true. Either:

gcloud config set container/use_client_certificate True

or

export CLOUDSDK_CONTAINER_USE_CLIENT_CERTIFICATE=True

I was using the GOOGLE_APPLICATION_CREDENTIALS variable, but alas, was not working. I tried the "gcloud config set" option up above, but that also didn't work. Finally, I used the env variable CLOUDSDK_CONTAINER_USE_CLIENT_CERTIFCIATE and that actually finally did work.