Terraform gke node pool getting 403 on googleapi: Error 403: Required “container.clusters.update”

Question

I'm getting that 403 error when terraform, v0.11.11, is applying to the node pool that is managed separately from the gke cluster creation.

Full error:

google_container_node_pool.np: error creating NodePool: googleapi: Error 403: Required "container.clusters.update" permission(s) for "projects//locations/us-central1/clusters/". See https://cloud.google.com/kubernetes-engine/docs/troubleshooting#gke_service_account_deleted for more info., forbidden

I ran through the troubleshooting guide but all it says is to disable and then enable the api again, which I did try, and still am getting that error.

Also using the google and google beta providers both versions 1.20

Answer 1

try to delete the default GKE service account, and re enable the service using the gcloud command which is going to recreate the default service account. If that's not working for you try to change the roles of the account to "Editor" or create a custom role including the "container.clusters.update" permission.

Answer 2

So the root cause was I was using a custom module but passing credentials down to the custom module in the module block, but it was still using the original credentials that were being used to test the custom module. Once I changed the custom module creds to what they should be it was working.

Answer 3

I too ran into the same problem. Looks like the issue is that the google_container_node_pool resource is trying to update cluster in the project specified in the terraform google provider block rather than the project in which the actual cluster exits. I was able to fix it by specifying the same project provided in the google_container_node_pool which is same as the google_container_cluster resource.