Listing GCS bucket blobs from Cloud Function in the same project

Question

I want to list files in a bucket from a Google Cloud Function in the same project.

Right now, my code for the Cloud Function is

main.py

from google.cloud import storage

def my_function(event, context):
    storage_client = storage.Client()

    blobs = storage_client.list_blobs("my_bucket")

    for blob in blobs:
        print(blob.name)

requirements.txt

google-cloud==0.34.0

The error I'm getting is

from google.cloud import storage ImportError: cannot import name 'storage' from 'google.cloud' (unknown location)

I was under the impression that google-cloud would be pre-loaded on the machines running the Cloud Function but it seems like this is not the case.

I have tried:

Reading the Google docs which is where the code comes from
Reading other questions here, namely this (this one suggested that google-cloud is not pre-loaded which is why I added it to the requirements.txt) and this (but I think there is a way to do this without setting GOOGLE_APPLICATION_CREDENTIALS, as I'm calling from within the same project). Also read this one but the approved answer's code looks like mine, which isn't working.

How can I list the files on the bucket?

That's a strange error. It's not the usual ModuleNotFoundError: No module named 'foo' error you get when a module isn't installed. Googling a bit, that error seems to come up when you're trying to import a module from your own source tree and you don't quite have it right. I'm not sure how this applies to your project or Google Cloud in general. — CryptoFool

Iñigo González Iñigo González · Accepted Answer · 2020-09-20T21:18:21

The google-cloud module (as is) is deprecated. Last version is from 2018.

You need to specify just google-cloud-storage in your requirements.txt, like this:

google-cloud-storage>=1.31.0

Both pypy and the google-cloud-python github page, say you need to import just the modules for the service you will use.

As today (sep 2020), when you install just google-cloud-storage, it ends up installing google-cloud as a depency but there is no need to use it or import it from your code.

Also, remember that the service account that executes the cloud function must have the 'storage.buckets.list' IAM permission.

Listing GCS bucket blobs from Cloud Function in the same project

2 Answers