Permission denied writing from App Engine to associated Cloud Storage bucket

Question

We have a project that runs on App Engine and creates files on Cloud Storage. The two are connected as being part of the same cloud platform project.

In App Engine we have a "Google APIs Console Project Number", and in Cloud Console -> Credentials we have that project number listed under "Client ID" (1[..........].apps.googleusercontent.com) and "Email Address" (1[..........]@developer.gserviceaccount.com).

Every morning, we have some cron jobs that upload files to our Cloud Storage bucket. This has worked flawlessly since September 2013 but as of this morning (Oct 16, 2014) we're getting "permission denied" errors from Cloud Storage.

We're using the cloudstorage client library, which raises cloudstorage.ForbiddenError. Here's the log & exception output:

Expect status [201] from Google Storage. But got status 403.
Path: u'/bucketname/icon_20141016.png'.
Request headers: {'x-goog-resumable': 'start', 'x-goog-api-version': '2', 'content-type': 'image/png', 'accept-encoding': 'gzip, *'}.
Response headers: {'alternate-protocol': '443:quic,p=0.01', 'content-length': '151', 'via': 'HTTP/1.1 GWA', 'x-google-cache-control': 'remote-fetch', 'vary': 'Origin', 'server': 'UploadServer ("Built on Oct 9 2014 15:35:27 (1412894127)")', 'date': 'Thu, 16 Oct 2014 11:56:10 GMT', 'content-type': 'application/xml; charset=UTF-8'}.
Extra info: None.

Since we're using the Cloud platform connection between the two services, I feel like I can only diagnose the problem on my production App Engine instance. I would prefer not to deploy new versions and risk breaking a production server. This also appears to be a Cloud Storage issue this morning, but the only status page I could find says everything is working fine.

Is it possible the ACL on the bucket has changed? You can check with gsutil getacl gs://<bucket-name>. — tx802
@tx802 Something changed somewhere... not sure where, but I've resolved it. Will post an answer with details. — mrb

mrb mrb · Accepted Answer · 2014-10-16T12:59:12

As @tx802 suggested, I checked the bucket ACLs carefully.

$ gsutil getacl gs://bucket

    <Entry>
        <Scope type="UserByEmail">
            <EmailAddress>1[..........]@developer.gserviceaccount.com</EmailAddress>
        </Scope>
        <Permission>FULL_CONTROL</Permission>
    </Entry>

I looked at the App Engine application settings and saw the service account is actually [email protected], so I gave that account full control:

$ gsutil chacl -u [email protected]:FC gs://bucket

I'm not sure what changed since yesterday's cron run, but now it succeeds.

Permission denied writing from App Engine to associated Cloud Storage bucket

1 Answers