BigQuery Data Transfer Service: frequency limit exceeded

1
votes

I want to create a Data Transfer from AWS S3 to BigQuery with an every 12 hours Repeats Custom Schedule but get the following error:

Frequency limit exceeded. The required min_schedule_interval is [24h], the actual delay between [2021-03-15T14:45:00-07:00] and [2021-03-16T02:45:00-07:00] is [12h].

Setting the Custom Schedule to every 24 hours or more works fine, but any less produces this error.

According to the documentation for schedule options, "the minimum interval is 15 minutes".

Has anyone encountered this before or know how to resolve?

1
google-bigquery
Were you able to find a solution to this @KEllis? - Vaibhav Rathore

1 Answers

0
votes

I had the same error and checked the documentation.

"the minimum interval is 15 minutes". seems to be the case for Cloud Storage. In the case of AWS S3 integration, the minimum interval is 24 hours.

https://cloud.google.com/bigquery-transfer/docs/s3-transfer#limitations

The minimum interval time between recurring transfers is 24 hours. The default interval for a recurring transfer is 24 hours.