Big query is not loading GCS orc source

Question

I am trying to load an ".ORC" file present in GCS into Bigquery, however its throwing below error. I am seeing documentation and it says it's possible to load ORC data. Any input will be very helpful .

bq load --source_format=ORC mt-gd-sc-sct-bq-dev:stg_supply_chagn_tables.test_external1 gs://<bucket>/del/run_date=20200727050329/

Error: FATAL Flags parsing error: flag --source_format=ORC: value should be one of <CSV|NEWLINE_DELIMITED_JSON|DATASTORE_BACKUP|AVRO|PARQUET> Run 'bq.py help' to get help *

shollyman shollyman · Accepted Answer · 2020-07-27T15:48:19

That would seem to indicate you have a very old version of the BQ CLI tool installed.

From bq help load:

 --source_format: <CSV|NEWLINE_DELIMITED_JSON|DATASTORE_BACKU
                      P|AVRO|PARQUET|ORC>: Format of source data. Options
                      include:
                      CSV
                      NEWLINE_DELIMITED_JSON
                      DATASTORE_BACKUP
                      AVRO
                      PARQUET
                      ORC

Try running gcloud components update to update your cloud SDK installation?

Big query is not loading GCS orc source

1 Answers