How to easily switch gcloud / kubectl credentials

Question

At work we use Kubernetes hosted in GCP. I also have a side project hosted in my personal GCP account using Google App Engine (deploy using gcloud app deploy).

Often when I try to run a command such as kubectl logs -f service-name, I get an error like "Error from server (Forbidden): pods is forbidden: User "[email protected]" cannot list resource "pods" in API group "" in the namespace "WORK_NAMESPACE": Required "container.pods.list" permission." and then I have to fight with kubectl for hours trying to get it to work.

Can somebody please break it down for a slow person like me, how gcloud and kubectl work together, and how I can easily switch accounts so I can use gcloud commands for my personal projects and kubectl commands for my work projects? I'm happy to nuke my whole config and start from scratch if that's what it takes. I've found various kubectl and gcloud documentation but it doesn't make much sense or talks in circles.

Edit: this is on Linux.

Which operating system. Why not create two different logins and then switch terminal windows (if on Linux). For Windows, move your setup to batch scripts and open different Command Prompts for each environment. The first step is to define commands that correctly set up each environment. You should not have to struggle for hours to switch. — John Hanley
Have a gander at this post that I wrote. There are various ways that you can address your need: medium.com/google-cloud/… — DazWilkin
Two other features with kubectl : you can create multiple configs and then set KUBECONFIG per environment; you can include namespaces in contexts so you may not only default to a specific cluster and auth pair but a default namespace too. — DazWilkin
This one too on how gcloud works with kubectl: medium.com/google-cloud/… — DazWilkin

kaoskater08 kaoskater08 · Accepted Answer · 2020-02-26T15:24:22

I'm in the same boat as you - apps deployed in GKE for work and personal projects deployed in my personal GCP account.

gcloud stores a list of logged in accounts that you can switch between to communicate with associated projects. Take a look at these commands:

gcloud auth login
gcloud auth list
gcloud config set account
gcloud projects list

To work with a specific project under one of your accounts you need to set that configuration via gcloud config set project PROJECT_ID

kubectl has a list of "contexts" on your local machine in ~/.kube/config. Your current context is the cluster you want to run commands against - similar to the active account/project in gcloud.

Unlike gcloud these are cluster specific and store info on cluster endpoint, default namespaces, the current context, etc. You can have contexts from GCP, AWS, on-prem...anywhere you have a cluster. We have different clusters for dev, qa, and prod (thus different contexts) and switch between them a ton. Take a look at the [kubectx project][1] https://github.com/ahmetb/kubectx for an easier way to switch between contexts and namespaces.

kubectl will use the keys from whatever GCP account you are logged in with against the cluster that is set as your current context. i.e., from your error above, if your active account for gcloud is your personal but try to list pods from a cluster at work you will get an error. You either need to set the active account/project for gcloud to your work email or change the kubectl context to a cluster that is hosted in your personal GCP account/project.

How to easily switch gcloud / kubectl credentials

1 Answers