Bigquery loading error

1
votes

Recently I got a error when loading data into bigquery.

Error encountered during job execution: Total JSON data size exceeds max allowed size. Total size is at least: 18446744073709551615. Max allowed size is: 13194139533312.

It's impossible to have such big files size, my loading data is under one terabytes. I don't know where this error comes from.

1
google-bigquery
Are you loading a single file or multiple files in your load job? - DoiT International
Multiple files, about 800GB and this is a daily job, it never happen before. Recently I got this error several times and sometimes retry will succeed. The file size that big query show me is 1400PB. - Xiaomeng Yi

1 Answers

1
votes

This is likely related to a recent change to our JSON import pipeline. We've rolled back the change, so hopefully your import should now succeed.