Cloud dataflow job failed without reason

1
votes

I have a Google Cloud Dataflow job to be executed using the Apache Beam API (0.4.0). The pipeline runs successfully using a local runner. When I submit the job using the dataflow runner the job is submitted, however it fails after 32 seconds without displaying a reason anywhere. The logs appear to be empty. The gcloud cli isn't any help either:

$ gcloud beta dataflow logs list 2017-01-23_12_51_23-5463584243087329795
E 2017-01-23T21:51:53 2017-01-23_12_51_23-5463584243087329795_00000159cd197209 (cdfde4683948d134): Workflow failed.

How can I track down the cause of the error?

1
google-cloud-dataflowapache-beam

1 Answers

3
votes

This is because the Dataflow API has not been enabled for your project. Do step 3 here: https://cloud.google.com/dataflow/docs/quickstarts/quickstart-java-maven

Dataflow will have a better error message for this soon.