Dataflow API will not activate in GCP

0
votes

I am attempting to use the dataflow API in my project. In the cloud dashboard, I have enabled all the API's using the prescribed link: https://console.cloud.google.com/flows/enableapi?apiid=dataflow,compute_component,logging,storage_component,storage_api,bigquery,pubsub,datastore&redirect=https:%2F%2Fcloud.google.com%2Fdataflow%2Fgetting-started&showconfirmation=true&_ga=1.200330088.756201991.1447156694

However, when I run my dataflow job in eclipse, I get the runtime error:

Caused by: java.lang.RuntimeException: Failed to create a workflow job: Project has not enabled the API. Please use Google Developers Console to activate the 'dataflow' API for your project. at com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.run(DataflowPipelineRunner.java:572) at com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.run(DataflowPipelineRunner.java:190) at com.google.cloud.dataflow.sdk.Pipeline.run(Pipeline.java:180) at com.ri.gateway.storedata.CloudConnectPipeline.main(CloudConnectPipeline.java:146)

Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden { "code" : 403, "errors" : [ { "domain" : "global", "message" : "Project has not enabled the API. Please use Google Developers Console to activate the 'dataflow' API for your project.", "reason" : "forbidden" } ], "message" : "Project has not enabled the API. Please use Google Developers Console to activate the 'dataflow' API for your project.", "status" : "PERMISSION_DENIED" }

I am unsure why this would happen - as the API's seem active through the gcp dashboard?

1
google-app-enginegoogle-cloud-dataflow
Are you sure you are submitting the job against the same GCP project as you are checking the enabled APIs for in the Developers Console? If the Developers Console says the Dataflow API is enabled, you should not be getting this message. I'd suspect --project pipeline option in Eclipse is set incorrectly. Please note the distinction between project id, project name, and project number. You should use project id when setting --project pipeline option. – Davor Bonaci

1 Answers

2
votes

Please double-check that you are submitting the Cloud Dataflow job against the same Google Cloud Platform project, which you enabled the Cloud APIs on. This is different than the Eclipse project you are launching the job from.

If the Google Developers Console says the Cloud Dataflow API is enabled, most likely, the --project pipeline option in Eclipse is set incorrectly. Please note the distinction between project id, project name, and project number. Please use the project id, not the other GCP project identifiers.

Finally, we've occasionally seen that enabling the APIs via the link you've posted above didn't take effect. Almost always, re-trying shortly after solved the problem.