Best way to limit maximum executions of Azure durable functions activity per timespan

Question

Being rather new to the Azure Durable Functions landscape, I am struggling to find the best way to handle downstream calls to an API that has rate limits implemented.

The way my flow is set up, is like below:

• HistorySynchronizer_HttpStart function: has HttpTrigger and DurableClient bindings in the signature and calls the next orchestration function:
• HistorySynchronizer function: has OrchestrationTrigger binding in the signature. This function will make a call to the API (await) and get a collection back. For every item in that collection, it will start a new Activity: context.CallActivityAsync() (combining them in a List and perform a Task.WhenAll())
• ProcessActivity functions: has ActivityTrigger binding in the signature. This function will have to make a call to the rate-limited API endpoint. And it's these activities I want to throttle (across multiple orchestrations).

So, what I am looking for is an implementation to a throttling pattern:

• I need a shared state that is thread-safe (I was thinking about Durable Entity) that keeps track of the number of calls made to that API
• Before the ProcessActivity function makes the call to that API, it has to check with the Durable Entity if it can call the API, or if it has to wait a certain TimeSpan before performing that call.
• In case it has to wait, the Activity would have to 'sleep', and I was thinking about a Durable Timer, but it seems that should be used in an Orchestration function, instead of an Activity function.
• In case the call to the API is allowed, the Activity has to update the Rate counter in that shared state object.

I don't see a standard implementation to achieve this and I want to move as much of that Check/Sleep logic away from the main orchestration.

Would the best approach be to have a suborchestration implemented for every API call that has to be throttled, where the check has to happen, before the Activity is called?

Looking forward to any insights.

Answer 1

Sam, I see several options. I've also created a video as a response to this. Let's step back and see how we would do this using regular functions (not durable).

The first approach would be to turn the function into a queue-triggered function, and use the queue mechanism to control scale out, by using batchSize and newBatchThreshold:

The other way would be to have an http-triggered function and use this in the host.js file:

With durable functions we can do this:

You specifically asked regarding controlling scale-out per timespan, and this is how I would do this:

1. convert your activity function to be queue-triggered
2. upon every execution of the queue-triggered function, this function insert a record into table storage indicating that there's an active run
3. prior to actually triggering the external http call, ping the table storage to see how many active connections there are.
4. if there are under X number of current connections, then process the message (i.e. make the external call), if table storage shows that all the connections are taken, then put the message back on the queue!

Answer 2

There is a better way to do it. Instead of trying to limit it on your side based on concurrent activity functions or active http requests, why don't you rely on the API itself? It knows when it's time to return 429.

I would add a queue, grab a task, call the API, if there is 429, put the task message back into the queue with an exponential delay policy.