GCP dataflow quota issue

0
votes

I'm new in GCP plateform. I am setting up a dataflow job that collects data from pub/sub topic and send it to bigquery but i am having a quota issue. See below the error code. I am looking forward to having a recommandation. Please help.

    {
  "textPayload": "Project gold-apparatus-312313 has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/7 instances, 4/4 CPUs, 1230/818 disk GB, 0/250 SSD disk GB, 1/99 instance groups, 1/49 managed instance groups, 1/99 instance templates, 1/3 in-use IP addresses.\n\nPlease see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.",
  "insertId": "fjxvatcdw1",
  "resource": {
    "type": "dataflow_step",
    "labels": {
      "project_id": "76303513563",
      "region": "us-central1",
      "step_id": "",
      "job_id": "2021-05-07_16_07_09-4793338291722263008",
      "job_name": "my_job1"
    }
  },
  "timestamp": "2021-05-07T23:16:20.877954859Z",
  "severity": "WARNING",
  "labels": {
    "dataflow.googleapis.com/region": "us-central1",
    "dataflow.googleapis.com/log_type": "system",
    "dataflow.googleapis.com/job_id": "2021-05-07_16_07_09-4793338291722263008",
    "dataflow.googleapis.com/job_name": "my_job1"
  },
  "logName": "projects/gold-apparatus-312313/logs/dataflow.googleapis.com%2Fjob-message",
  "receiveTimestamp": "2021-05-07T23:16:21.809934742Z"
}
1
google-cloud-platformcloudgoogle-cloud-pubsubdataflow
Did you read the link provided in the error message? cloud.google.com/compute/docs/resource-quotas - John Hanley

1 Answers

1
votes

Your error code shows insufficient disk size quota (Required - 1230 Available-818).So hereby its recommended to increase your quota.

For increasing your quota,you can request an increase in quota.

To submit a GCE quota increase : https://support.google.com/cloud/answer/6075746

Reference:

Quotas & limits for Dataflow - https://cloud.google.com/dataflow/quotas