BigQuery load_table_from_storage won't recognize uri

1
votes

I'm running into the following error when trying to load a table from google cloud storage: BadRequest: 400 Load configuration must specify at least one source URI (POST https://www.googleapis.com/bigquery/v2/projects/fansidata/jobs)

Meanwhile my uri is valid (ie: i can see it in the gcs web app)

uris = ['gs://my-bucket-name/datastore_backup_analytics_2016_12_21_2_User/1569751766512529035929A5AA9742/output-0']

job_name = 'Load_User' 
destinationTable = dataset.table('Transfer')
job = bigquery_client.load_table_from_storage(job_name, destinationTable, uris)            
job.begin()
2
pythongoogle-bigquerygoogle-cloud-storage

2 Answers

2
votes

I could be wrong, but it looks like load_table_from_storage in the Python API expects a single string for the third argument as opposed to a list. If you want to match multiple files, you can use a * at the end. For example,

uri = 'gs://my-bucket-name/datastore_backup_analytics_2016_12_21_2_User/1569751766512529035929A5AA9742/output-*']

job_name = 'Load_User' 
destinationTable = dataset.table('Transfer')
job = bigquery_client.load_table_from_storage(job_name, destinationTable, uri)            
job.begin()
1
votes

Client.load_table_from_storage takes one or more source URIs (that is what *soure_uris means in python). E.g.:

job = client.load_table_from_storage(
    'load-job-123', my_table_object,
    'gs://my-bucket-name/table_one',
    'gs://my-bucket-name/table_two')