Webpage on localhost cannot accessing Google Cloud Storage bucket file.csv

Question

Not able to read this file: https://storage.googleapis.com/[bucket_name]/[filename].csv from webpage running on http://localhost:3000.

WHAT I AM TRYING TO DO:

This seems to be a simple and I would expect a common model. ML job on Google Cloud processes data and places the output in Google Cloud Storage Bucket as a .csv file. I have a react.js web application that contain d3.js visualization and the data for this viz is to come from the .csv file that is the cloud bucket. The file access is 'no public'.

I set the CORS policy on the Google Storage Bucket as:

[
  {
    "origin": ["http://localhost:3000"],
    "method": ["GET"],
    "maxAgeSeconds": 3000,
    "responseHeader": ["Authorization", "Content-Range", "Accept", "Content-Type", "Origin", "Range","Access-Control-Allow-Origin"]
  }
]

I got service key json file.

I setup $env:GOOGLE_APPLICATION_CREDENTIALS "service_key_file_path.json" in local directory.

In the web application that is served on http://localhost:3000, I am trying to read the .csv file with this code and url (I replaced here bucket and file names with placeholders):

d3.csv("https://storage.googleapis.com/[bucket_name]/[filename].csv",row).then(data => { ... });

d3.csv() is just a wrapper for the fetch(). In addition of fetching the csv file it also converts data to json.

What else do I need to setup? Please help (using JavaScript).

From the call, I get status code: 403

Details are here:

GENERAL
Request URL: https://storage.googleapis.com/bucket_name/filename.csv
Request Method: GET
Status Code: 403 
Remote Address: 127.0.0.1:8888
Referrer Policy: strict-origin-when-cross-origin

RESPONSE HEADERS
access-control-allow-origin: http://localhost:3000
access-control-expose-headers: Accept, Access-Control-Allow-Origin, Authorization, Cache-Control, Content-Length, Content-Range, Content-Type, Date, Expires, Origin, Range, Server, Transfer-Encoding, X-GUploader-UploadID, X-Google-Trace
alt-svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
cache-control: private, max-age=0
content-length: 223
content-type: application/xml; charset=UTF-8
date: Thu, 15 Apr 2021 04:30:23 GMT
expires: Thu, 15 Apr 2021 04:30:23 GMT
server: UploadServer
vary: Origin
x-guploader-uploadid: ABg...3W4FQ

REQUEST HEADERS
:authority: storage.googleapis.com
:method: GET
:path: /bucket_name/file_name.csv
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
origin: http://localhost:3000
referer: http://localhost:3000/
sec-ch-ua: "Google Chrome";v="89", "Chromium";v="89", ";Not A Brand";v="99"
sec-ch-ua-mobile: ?0
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: cross-site
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36
x-client-data: E...J3L
Decoded:
message ClientVariations {
  // Active client experiment variation IDs.
  repeated int32 variation_id = [3300111, 3300134, 3300161, 3313321, 3318776, 3329330, 3329635, 3329704, 3329769];
}

When I insert this url into browser https://storage.googleapis.com/bucket_name/filename.csv, I get:

<Error>
<Code>AccessDenied</Code>
<Message>Access denied.</Message>
<Details>Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object.</Details>
</Error>

NEXT, I also tried to use this url: https://storage.cloud.google.com/[bucket_name]/[filename].csv

When I insert it into browser, I get a prompt to download the file.

When I call the url from the web app

d3.csv("https://storage.cloud.google.com/[bucket_name]/[filename].csv ",row).then(data => { ... });

I get CORS error origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

This is probably due to Google saying:

The authenticated browser download endpoint storage.cloud.google.com does not allow CORS requests. Note that the Cloud Console provides this endpoint for each object's public URL link. https://cloud.google.com/storage/docs/cross-origin

Did you chak your URL validity? Can you access that file directly from Chrome? — Michael Rovinsky
Your javascript, on your browser, doesn't have access to your env vars and your service account key file stored on your computer (sandbox execution). Thus, it can get it and generate a valid credential. You can see that il the request header: you have no Authorization header with bearer token, so, it can't work. Share more pieces of your code, and explain how it will run at the end to get a solution from the community. — guillaume blaquiere
Hi guillaume, I made some updates to my case above to be more clear. Please take a look. I suspect it is related to Authorization header but don't know how to type it into the function call. Is the bearer token the private key (where do I get the token)? — psabela

afarre afarre · Accepted Answer · 2021-06-01T15:13:12

As the definition for each field states:

Public URL (https://storage.googleapis.com/...):

Anyone with this link can access the object on the public internet

Authenticated URL (https://storage.cloud.google.com/...):

Only users granted permission can access the object with this link

Therefore decide which one suits best your needs.

Regarding the error Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object.: Anonymous user means that no user is being passed in the call being made to Cloud Storage. Please read this post and follow the steps.

Webpage on localhost cannot accessing Google Cloud Storage bucket file.csv

1 Answers