Download entire directories from google cloud storage bucket using python

0
votes

I have a google cloud storage bucket, I can download objects using the download_blob function in python, I can also use the gsutil cp or gcloud compute scp function to download entire directories.

Is there a way to download an entire directory from the storage bucket using python as a single zip file.

doing it this way Python - download entire directory from Google Cloud Storage requires me to download file by file.

Is there a way to download an entire directory at once ?

1
pythongoogle-cloud-platformgoogle-cloud-storage
Any method you use, it will still be "file by file". Because one download request needs to be made per file. I think the linked answer will work for you just fine – OneCricketeer

1 Answers

0
votes

Cloud Storage has no concept of "directories" -- instead, each blob can be given a name that can resemble a directory path. Downloading entire "directories" is the same as downloading all blobs with the same prefix.

This means that you can use wildcards with gsutil:

gsutil -m cp gs://bucket/data/abc/* .

would copy every blob whose name starts with /data/abc/.