Azure Durable Functions fan-out scale limit?

Question

I have a durable function orchestrator that fans-out into multiple activity functions to handle some workload. The following code is an example where Function_2 is the one that fans out to handle the workload:

public static async Task Run(DurableOrchestrationContext ctx)
{
    // get a list of N work items to process in parallel
    object[] workBatch = await ctx.CallActivityAsync<object[]>("Function_1");

    var parallelTasks = new List<Task<int>>();
    for (int i = 0; i < workBatch.Length; i++)
    {
        Task<int> task = ctx.CallActivityAsync<int>("Function_2", workBatch[i]);
        parallelTasks.Add(task);
    }

    //How many instances of Function_2 will handle the workload?
    await Task.WhenAll(parallelTasks);

    // aggregate all N outputs and send result to Function_3
    int sum = parallelTasks.Sum(t => t.Result);
    await ctx.CallActivityAsync("Function_3", sum);
}

My question is how many instances of Function_2 will be spawned in order to handle the work. I know that it depends on the number of Tasks, so lets say I have 5000 tasks. I doubt it would spawn 5000 instances, but what is the upper limit and can I control it. I read through the documentation multiple times but was unable to find information on this subject. I know that by definition I should not care about that as it is handled for me, however my Tasks can overload a backing resource they all depend upon.

Mikhail Shilkov Mikhail Shilkov · Accepted Answer · 2018-03-08T17:10:06

Behind the scenes, each CallActivity call becomes a message in a Storage Queue, so 5000 messages in your example. The messages will there be consumed by Function App.

It will run multiple invocations in parallel, but of course not all at the same time. You won't see the exact numbers anywhere in the docs, since they will be defined by internal scale controller logic. They will also depend on duration of each activity call, it's CPU usage etc.

The results may change over time too.

So, your milage may vary, and you should test your scenario.

Azure Durable Functions fan-out scale limit?

1 Answers