Google Analytics realtime API raises Rate Limit Exceeded on Cloud Functions

Question

I'm using the google-api-python-client to create a Cloud Function that make one request to the Google Analytics Realtime API every minute. It's running fine for a long time.

This is the code used:

from googleapiclient.discovery import build
service = build('analytics', 'v3', credentials=credentials)

service.data().realtime().get(
  ids=f'ga:{view_id}',
  metrics='rt:pageviews',
  dimensions='rt:deviceCategory,rt:minutesAgo',
  quotaUser='my-function'
).execute(num_retries=3)

Today since 10AM UTC-3 I'm facing this error:

Quota Error: Number of recent failed reporting API requests is too high, please implement exponential back off.

According to the docs the limit is:

• 50,000 requests per project per day, which can be increased.
• 10 queries per second (QPS) per IP address

In the developer console, the Google Analytics API metrics has 25k requests in the last 7 days, an average of 0.05/s and a peak of 0.20/s when the error started.

The functions was deployed in us-east1. I deployed other instance in us-east4 and worked well. So I think the limit is in the IP address used in the us-east1, shared with other users.

There a way to isolate the quota for my project? I tried the quotaUser parameter, but seems has no effect.

Update: The region us-east4 started to raises the same error, so I switched to us-central1 which is working fine for now.

Answer 1

Update 23 dec, 16:00 UTC: The issue returned. API requests are returning the same error again.

---

Update 22 dec, 18:56 UTC: Since 1 hour ago, the Reporting API started responding again on queries. Since ~15 minutes ago the Realtime API is also responding again. Before they started working, the calls didn't return an error any more, instead the requests kept loading 'forever'.

---

Since 6 hours ago I'm randomly experiencing the same issue when using the Google Analytics API directly from my own server (in Berlin). I already tried the following things to work around the issue:

• Disable all API calls for an hour (didn't work)
• Authenticate again; new access_token and refresh_token (didn't work)

Why I highly suspect this is an issue on Google Analytics side (and you/me can't really fix this):

• My code stopped working randomly around 18:00 (local time - now 6h ago), this code has been running untouched for at least 4 years already.
• Fetching new access tokens works, using them doesn't (instant rate limiting)
• I checked the Google Developer Console page quota error graph and 0 errors are shown (...)
• Within the same console you can see the quota's that are configured for your project: I have a max of 2.000 requests per user per 100 seconds. I checked my historical data and I only run ~1.500 requests PER DAY.
• The error I'm getting is: HTTP 403 with as reason 'rateLimitExceeded'. According to the documentation this: "Indicates that the project queries per 100 seconds rate limits have been exceeded." Source: https://developers.google.com/analytics/devguides/reporting/core/v3/errors
• When I run the exact same query in the Google API Explorer tool, I get an expected HTTP 200 response. For example: https://developers.google.com/analytics/devguides/reporting/realtime/v3/reference/data/realtime/get

The exact error I'm getting back from the Google Analytics API:

"error": {
    "code": 403,
    "message": "Quota Error: Number of recent failed reporting API requests is too high, please implement exponential back off.\nSee developer documentation for details at https://developers.google.com/analytics/devguides/reporting/core/v4/limits-quotas#reporting_apis_request_errors",
    "errors": [
      {
        "message": "Quota Error: Number of recent failed reporting API requests is too high, please implement exponential back off.\nSee developer documentation for details at https://developers.google.com/analytics/devguides/reporting/core/v4/limits-quotas#reporting_apis_request_errors",
        "domain": "usageLimits",
        "reason": "rateLimitExceeded"
      }
    ]
  }

Answer 2

We experience the same and it seems to appear and resolve every now and then without any signal or signs from Google Analytics. It started the 21st of december. We tried decreasing the amount of calls (once a minute to less a minute), but that doesn't seem to help.

We are using v3 of the realtime API, but we did see some reference to the V4-api in the error logging:

https://developers.google.com/analytics/devguides/reporting/core/v4/limits-quotas#reporting_apis_request_errors

Maybe this can help you finding the cause.

Answer 3

I have the same problem. I think the main cause is more requests timeout on google server side(Try the API Explorer with 5-10 requests). All requests that timeout are counted as FAILED and then you get: Quota Error: Number of recent failed reporting API requests is too high, please implement exponential back off ....

I saved some logs and i've seen some strange things: most of the "timeout requests" still receive the response after 144s, 240s or 480s. I still don't know what these magic numbers are but it looks like a pattern.