How can we load files under folder into Big Query from Google Storage?

0
votes

I have a folder of files and subdirectories in the google storage. Want to upload all the files under this folder into bqdataset.bqdatatable using a schema.

If I specify a single file it successfully uploads the file but while giving a folder path it is unable to do that.

Please advise.

Regards.

1
google-bigquery

1 Answers

2
votes

It must work! You should use below as a source

gs://your_bucket/your_folder/*

of course assuming all files have the same schema (which you should also provide) and you load your folder into one table

it works both using BigQuery UI or programmatically with API https://cloud.google.com/bigquery/docs/reference/v2/jobs#configuration.load