Google Bigquery load data with local file size limit

Question

Is there any limit on Google Bigquery load data with a local file with API?

As Google Bigquery document mention regarding Web UI, local file size is than <=10 MB and 16,000 Rows. Is the same limit will apply to API?

Answer 1

There are no special limits for local file uploads, 10MB and 16000 rows is only for UI. But I don't recommend uploading huge local files.

Answer 2

There is no BigQuery API to load local files. Local file load via bq command or Web UI - and I believe what happened when you do this - it is just upload file to GCS on your behalf and after this just doing normal API load job from GCS - you can see it clearly in UI. But because Google want to have reasonable user experience from WebUI/bq command - additional much more strict limits are here for upload "local" files.

My recommendation to go GCS path to load big files (https://cloud.google.com/bigquery/docs/loading-data-cloud-storage) Important thing - it is free (compare with streaming where you will pay for streamed data)

    Limits are following (from https://cloud.google.com/bigquery/quotas)
    Load jobs per table per day — 1,000 (including failures)
    Maximum columns per table — 10,000
    Maximum size per load job — 15 TB across all input files for CSV, JSON, and Avro
    Maximum number of files per load job — 10 Million total files including all files matching all wildcard URIs
    For CSV and JSON - 4 GB compressed file, 5TB uncompressed