I have a firestore backup file in GCS with the name: all_namespaces_kind_Rates.export_metadata. I have set up a cron job to update this file every 24 hours. What I need now is to find a way to programmatically send this export_metadata file to BigQuery. BigQuery has the capability of scheduling data transfer from GCS, but only for files with format: CSV, JSON, AVRO, PARQUET AND ORC. How can I transfer my firestore backup files programmatically into BigQuery?
1 Answers
1
votes
If your cron job can access bq command line tool, have you tried:
bq load --source_format=DATASTORE_BACKUP [DATASET].[TABLE] [PATH_TO_SOURCE]
See more about the command: https://cloud.google.com/bigquery/docs/loading-data-cloud-firestore#loading_cloud_firestore_export_service_data
