How to access GCS bucket objects across projects using service account?

Question

I created a Google App Engine app that listens for Google Cloud Storage notifications and whenever a new object is created on GCS, the app needs to open the new object and perform operations based on its contents. I can't access the object contents when the app and the gcs bucket are in different projects.

Configuration:

I have created a service account in project A with Storage Object Admin permissions, associated the GAE app with it, activated the service account using:

gcloud auth activate-service-account [ACCOUNT] --key-file=KEY_FILE

I then created a bucket gs://some_bucket in project B in the same region as my GAE app, and added my service account as an owner of the bucket.

I added my service account as a member of project B with "Storage Object Admin" permissions.

I created a watchbucket channel between my application and the bucket using

gsutil notification watchbucket -i [ChannelId] -t [Token] https://[app-name].appspot.com/ gs://some_bucket

My application is now receiving post requests, I can parse through them, find the source bucket, the size, object name, etc. but I can't read the objects themselves. I get the following error.

{Location: ""; Message: "Access Denied: File gs://some_bucket/some_object: Access Denied"; Reason: "accessDenied"}

I tested the above configuration within the same project (project A), and I am able to read the objects and operate on them. This is a permissions issue that I can't figure out.

The service account in project A needs read access to the contents of the buckets and objects. Can you add is as a member to project B? — Brandon Yarbrough
Yea I already added the service account as a member with "Storage Object Admin" permissions to project B. I also added the service account as a bucket owner to the source_bucket. — ioverzero
Owning the bucket doesn't imply read access to the object. Try explicitly granting the service account read permission on that specific object. — Brandon Yarbrough
I understand, that makes sense, it works when I explicitly set the service account as an owner of an object, is there a way to automate this so that all future objects are readable by the service account? — ioverzero
Yes. Buckets have a "default object ACL" property, which specifies the ACL that any new object will have unless otherwise specified. — Brandon Yarbrough

ioverzero ioverzero · Accepted Answer · 2017-03-27T20:44:58

GCS Bucket permissions are different than GCS object permissions, being a bucket owner does not translate into object owner or having object access. You can grant read permissions to all existing GCS objects in your bucket recursively using the following:

gsutil -m acl ch -u [email protected]:R -r gs://example-bucket

which will recursively grant the service account read permission to all objects in the bucket.

One might also want to change the bucket object default permissions so that all future objects coming into your GCS bucket have the desired permissions

gsutil defacl ch -u [email protected]:READ gs://example-bucket

Changing object ACL's: https://cloud.google.com/storage/docs/gsutil/commands/acl

Changing default object ACL's: https://cloud.google.com/storage/docs/gsutil/commands/defacl

How to access GCS bucket objects across projects using service account?

1 Answers