Dataflow job failed - BigQuery import job not found

0
votes

Another one of our pipelines failed. Appears to having trouble writing to BigQuery again.

id: 2015-12-04_08_01_48-5133681620861162960

This is our 3rd pipeline to fail when either reading or writing from BigQuery. Is there an issue currently?

Workflow failed. Causes: (9e23278dca02e8a7): BigQuery import job "dataflow_job_5133681620861162507" failed. Causes: (9e23278dca02ee74): BigQuery creation of import job for table "events_2015_12_05_denormalized" in dataset "PROJECT_MALBEC_DENORMALIZATION" in project "<removed>" failed. Causes: (9e23278dca02e441): Error:
 Message: Not found: Job roy-morgan-ua-model:dataflow_job_5133681620861162507
 HTTP Code: 404
1
google-bigquerygoogle-cloud-dataflow
none per status.cloud.google.com - Mikhail Berlyant
Thanks Sam. I've just flicked you guys a summary email (dataflow feedback) of the issues we're experiencing. - Graham Polley
I am having the same issue currently @SamMcVeety Can you help me with it? - Anuj

1 Answers

2
votes

This is a known issue when Dataflow retries a BigQuery job after a timeout. We're actively working on a fix.