Loading data from Bigquery to google storage bucket in CSV file format

3
votes

I run a dataset in bigquery on a daily basis which i need to export to my google storage bucket. The dataset is greater than 10MB which means i'm unable to use app-scripts.

Essentially, I'd like to automate a data load using my bigquery script which exports the dataset as a CSV file to google storage.

Can anyone point me into the right direction in terms of which programme/method to use. Please also share your experiences.

Thanks

1
google-bigquerygoogle-cloud-storageetl

1 Answers

3
votes

Here you can find some details on how to export data from BigQuery to Cloud Storage along with a sample written in Python.

https://cloud.google.com/bigquery/exporting-data-from-bigquery

You can implement a simple application running on App Engine that will contain cron job scheduled to run once a day and perform the steps described in the tutorial above.

https://cloud.google.com/appengine/docs/python/config/cron