Waiting for an azure function durable orchestration to complete

Question

Currently working on a project where I'm using the storage queue to pick up items for processing. The Storage Queue triggered function is picking up the item from the queue and starts a durable orchestration. Normally the according to the documentation the storage queue picks up 16 messages (by default) in parallel for processing (https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue), but since the orchestration is just being started (simple and quick process), in case I have a lot of messages in the queue I will end up with a lot of orchestrations running at the same time. I would like to be able to start the orchestration and wait for it to complete before the next batch of messages are being picked up for processing in order to avoid overloading my systems. The solution I came up with and seems to work is:

public class QueueTrigger
    {
        [FunctionName(nameof(QueueTrigger))]
        public async Task Run([QueueTrigger("queue-processing-test", Connection = "AzureWebJobsStorage")]Activity activity, [DurableClient] IDurableOrchestrationClient starter,
            ILogger log)
        {
            log.LogInformation($"C# Queue trigger function processed: {activity.ActivityId}");
            string instanceId = await starter.StartNewAsync<Activity>(nameof(ActivityProcessingOrchestrator), activity);            

            log.LogInformation($"Started orchestration with ID = '{instanceId}'.");

            var status = await starter.GetStatusAsync(instanceId);
            do
            {
                status = await starter.GetStatusAsync(instanceId);
            } while (status.RuntimeStatus == OrchestrationRuntimeStatus.Running || status.RuntimeStatus == OrchestrationRuntimeStatus.Pending);
        }

which basically picks up the message, starts the orchestration and then in a do/while loop waits while the staus is Pending or Running. Am I missing something here or is there any better way of doing this (I could not find much online). Thanks in advance your comments or suggestions!

PramodValavala-MSFT PramodValavala-MSFT · Accepted Answer · 2021-08-04T06:47:58

This might not work since you could either hit timeouts causing duplicate orchestration runs or just force your function app to scale out defeating the purpose of your code all together.

Instead, you could rely on the concurrency throttles that Durable Functions come with. While the queue trigger would queue up orchestrations runs, only the max defined would run at any time on a single instance of a function.

This would still cause your function app to scale out, so you would have to consider that as well when setting this limit and you could also set the WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT app setting to control how many instances you function app can scale out to.

Waiting for an azure function durable orchestration to complete

1 Answers