I'm looking to move all S3 assets to Google Cloud for a bunch of reasons. However, I have ~25 buckets, with thousands of files in each. I'm aware of the Google Storage Transfer tool - https://cloud.google.com/storage/transfer/getting-started - but that only works on buckets one at a time. Is there anything to do all of them at once?
1
votes
1 Answers
2
votes
The Google Cloud Storage Transfer service is still your best bet, especially if your buckets are very, very large.
If your buckets aren't large enough to bother setting it up, you could use the gsutil command-line tool with a little bit of scripting to accomplish this:
for bucket in bucket1 bucket2 bucket3 bucket4 etc; do
gsutil -m cp -r s3://$bucket/* gs://$bucket
done
