Can we access gsutil from Google Cloud Dataflow? if yes, then could someone please explain with example?

Question

I'm new with google cloud dataflow and I need to copy files in GCS (Google Cloud Storage) from one bucket to another and rename. Answer with example would highly appreciated.

Answer 1

Technically yes you can do this, but it would be better if you used Dataflow to just pick up the files and move them over to the new bucket instead of as a wrapper around gustil.

The class you need should be beam.io.gcsio.GcsIO() and that should be able to pick up and put down files where you need it too.