2
votes

When I use https://developers.google.com/bigquery/docs/reference/v2/tables/insert#try-it to upload a JSON file with not all fields inserted in all columns, it is accepted and I get nulls in my data.

When I use bq load and have a schema with mode nullable on possibly missing columns, the same file is rejected because there are different numbers of columns than in the schema.

If I add those columns to the JSON and set them to have a value null, I get an error because bq can't convert that to a string.

If I add those columns to the JSON and set them to have a value '' then the upload proceeds just fine..but now I have empty strings rather than null. Which is not what I want.

How can I tell the bq tool to let me upload nulls as nulls into a nullable column?

2

2 Answers

3
votes

This is a bug, a fix should be out soon.

0
votes

To import JSON with bq:

Create the table with:

bq mk mytemptable schema.json

Load data using the JSON flag:

bq load --source_format=NEWLINE_DELIMITED_JSON mytemptable users.json