GCS write access from inside a GKE pod

5
votes

I am not able to get write access to a GCS bucket from within a GKE pod.

I have a GKE pod running. I have not changed any k8s configuration regarding service accounts. I have docker exec'd into the pod and installed gcloud/gsutil. gcloud auth list shows a [email protected] entry. From within GCS I have added that same account as storage admin, storage legacy bucket owner, storage object creator (i.e., I just tried a bunch of stuff). I am able to run gsutil ls gs://bucket. However when running gsutil cp file gs://bucket, it prints:

AccessDeniedException: 403 Insufficient OAuth2 scope to perform this operation. 
Acceptable scopes: https://www.googleapis.com/auth/cloud-platform

gsutil acl get gs://bucket prints:

AccessDeniedException: Access denied. Please ensure you have OWNER permission on gs://bucket

Other things I have tried are adding the allUsers and allAuthenticatedUsers as creators and owners of the bucket, with no change. I am able to write to the bucket from my dev machine just fine.

When I run gsutil acl get gs://bucket from another machine, it prints the same address as an OWNER as the output from gcloud auth list from within the pod.

What is the special sauce I need to allow the pod to write to the bucket?

3
google-cloud-storagegoogle-kubernetes-engine

3 Answers

2
votes

The GKE cluster was created with default permissions, which only has read scope to GCS. Solutions:

  1. Apply advice from Changing Permissions of Google Container Engine Cluster
  2. Set GOOGLE_APPLICATION_CREDENTIALS as described in https://developers.google.com/identity/protocols/application-default-credentials
0
votes

You need to set permissions for cluster (or better for particular node in case of Terraform):

    oauth_scopes = [
      "https://www.googleapis.com/auth/devstorage.read_write", // 'ere we go!
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
      "https://www.googleapis.com/auth/service.management.readonly",
      "https://www.googleapis.com/auth/servicecontrol",
      "https://www.googleapis.com/auth/trace.append",
      "https://www.googleapis.com/auth/compute",
    ]
0
votes

Had the same issue, I had to recreated a node pool with custom security config in order to get that access. Also, in my pod I mounted the SA provided in a secret (default-token-XXXXX) Then, once gcloud is installed in the pod (via docker file) works like a charm. The key is the node-pool config and mounting the SA.