I have a JSON payload like this that I’d like to stream insert into a table I’m managing:
{ “Field1”: “value1”, “Field2”: [“value1”, “value2”], }
Sometimes field2 is not present at all in the JSON. I thought repeated fields with type “STRING” could be null but I’m getting insert errors from the API saying it cannot be empty.
I’m doing the loading from python so can transform if necessary, I’m just not sure the best way to set the schema and JSON payload.
I think I have been able to reproduce your issue using the method insert_rows, I’m going to assume that that is the method you used. The error returned by that method was [1]. Please let me know if you are accessing the API in a different way or getting a different error.
There seems to be an issue in that method of the library, since REPEATED fields are not being treated the way we would expect.
I have filled an issue regarding this problem [2].
As a workaround I recommend using the method insert_rows_json [3]. This method has the same functionality as insert_rows but not its issues.
[1] {'index': 0, 'errors': [{u'debugInfo': u'', u'reason': u'invalid', u'message': u'Field value cannot be empty.', u'location': u'field2'}]}
[2] https://github.com/googleapis/google-cloud-python/issues/9602
