Getting HTTP transport error while executing google dataflow job

1
votes

I am getting a constant error while executing a Dataflow job:

BigQuery import job "dataflow_job_838656419" failed., : BigQuery creation of import job for table "TestTable" in dataset "TestDataSet" in project "TestProject" failed., : BigQuery execution failed., : HTTP transport error: Message: Invalid value for: String is not a valid value HTTP Code: 400

It does not give any specific reason for the google Dataflow job failing continuously. How do I know what is the error I am committing while executing the google Dataflow job?

1
google-cloud-dataflow
Can you provide the job ID please? - Tudor Marian
The job Id is 2016-08-29_10_19_38-10605284450672758855 - abhishek jha
Is the code using TableRow (developers.google.com/resources/api-libraries/documentation/…) and if so, does it use setF or set? - Tudor Marian
Yes the code uses TableRow object and uses set() - abhishek jha
It looks like the job is specifying "String" as the type of a schema field. BigQuery documentation says it should be spelled out as "STRING". Could you try it this way? - jkff

1 Answers

4
votes

The issue is the incorrect use of the BigQuery API, which is case-sensitive with respect to field type. Please specify "STRING" as the field type in the schema that you're providing.

Please see https://cloud.google.com/bigquery/docs/reference/rest/v2/tables for more details.