Azure Logic App with Azure Blob Storage Action: Getting 429 statusCode error

Question

I am using Azure Logic App with Azure BLOB Storage trigger.

When a blob is updated or modified in Azure Storage, I pull the content of blob created or modified from Storage, do some transformations on data and push it back to Azure Storage as new blob content using Create Content - Azure Blob Storage action of LogicApp.

With large number of blobs inserted (for example 10000 files) or updated into blob storage, Logic App gets triggered multiple runs as expected for these inserted blobs, but the further Azure Blob Actions fail with following error:

{
  "statusCode": 429,
  "message": "Rate limit is exceeded. Try again in 16 seconds."
}

Did someone face similar issue in Logic App? If yes, can you suggest what could be the possible reason and probable fix.

Thanks

Answer 1

Seems like you are hitting the rate limits on the Azure Blob Managed API.

Answer 2

Please check this doc: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-request-limits

For each Azure subscription and tenant, Resource Manager allows up to 12,000 read requests per hour and 1,200 write requests per hour.

you can check the usage by:

response.Headers.GetValues("x-ms-ratelimit-remaining-subscription-reads").GetValue(0)

or

response.Headers.GetValues("x-ms-ratelimit-remaining-subscription-writes").GetValue(0)

Answer 3

Please refer to Jörgen Bergström's blog about this: http://techstuff.bergstrom.nu/429-rate-limit-exceeded-in-logic-apps/

Essentially he says you can setup multiple API connections that do the same thing and then randomize the connection in the logic app code view to randomly use one of those connection which will eliminate the rate exceeding issue.

An example of this (I was using SQL connectors) is see below API connections I setup for my logic app. You can do the same with a blob storage connection and use a similar naming convention e.g. blob_1, blob_2, blob_3, ... and so on. You can create as many as you would like, I created 10 for mine:

You would then in your logic app code view replace all your current blob connections e.g.

@parameters('$connections')['blob']['connectionId']

Where "blob" is your current blob api connection with the following:

@parameters('$connections')[concat('blob_',rand(1,10))]['connectionId']

And then make sure to add all your "blob_" connections at the end of your code:

"blob_1": {
                    "connectionId": "/subscriptions/.../resourceGroups/.../providers/Microsoft.Web/connections/blob-1",
                    "connectionName": "blob-1",
                    "id": "/subscriptions/.../providers/Microsoft.Web/locations/.../managedApis/blob"
                },
"blob_2": {
                    "connectionId": "/subscriptions/.../resourceGroups/.../providers/Microsoft.Web/connections/blob-2",
                    "connectionName": "blob-2",
                    "id": "/subscriptions/.../providers/Microsoft.Web/locations/.../managedApis/blob"
                },
...

The logic app would then randomize which connection to use during the run eliminating the 429 rate limit error.