I've been importing data into my Cloud SQL instance using the gcloud import tool. I've broken up the data into smaller chunks to avoid the 24 hour limitation on imports. (Each import has about 16 million rows grouped into about 16 thousand import statements.) I completed two of the imports (each taking about 2 hours) and was importing the third:
gcloud sql instances import --database mydb mydb-1 gs://data-1/olddata/olddata-2010.sql.gz
After running for about 90 minutes, it stopped and reported the error
ERROR: (gcloud.sql.instances.import) Daily Limit Exceeded
I've found references to a 24 hour limit per import, but no other "daily limit" documented related to imports that I can see.
If I look at the API portion of the console, I do see that the Google Cloud SQL API has a daily limit of 10,000 requests/day, and that I have exceeded this. I can't reconcile this with my use of any of the Cloud SQL tools - either I would have had to do far more than 10000 requests, or far fewer. Looking at the Cloud SQL API reference documentation for v1beta4 the Instances.import documentation doesn't say it should take more than a single request to run (in fact, it doesn't say anything at all about quota). There don't appear to be any daily limits associated with retrieving from Cloud Storage.
- What is the daily limit?
- Where is the daily limit documented?
- Is there a way to get this limit increased?
- How many API calls to the Cloud SQL API are actually done for an import?
- Is there a way to do the import without running into this limit? (I have about 10 more data sets to import and would rather not have to space them out.)
