Transfer CSV To Google Cloud Storage

Question

I have a csv hosted on a server which updates daily. I'd like to setup a transfer to load this into Google Cloud Storage so that I can then query it using BigQuery.

I'm looking at the transfer service and it doesn't seem to have what I need e.g. only accepts csvs or files from other google storage buckets or amazon s3 buckets.

Thanks in advance

Answer 1

You can also use a URL to a TSV file as explained here and configure the transfer to run daily at the time of your choice.

Alternatively, if it still doesn't fit your need, you may install gsutil on your remote machine and use the gsutil rsync command and schedule it to run daily.