Azure functions only max concurrent at the time

2
votes

I have a Azure Function that handles SharePoint operations. Due to throttling I only want 10 concurrent functions running at the time and always 10 functions concurrently. I was considering to use functions with an Azure Service Bus Queue. Is there any build in the Azure platform to achieve this? Service Bus is not a requirement, so if other hubs or queues are better for this. I have looked in the Azure UI and havent found anything on the service bus side or function.

Test and observations:

I created a test with no success. I have created and deployed a functions that sleeps for 20 secs and then write a document to Cosmos DB:

[FunctionName("SleepFunction")]
public static void Run([ServiceBusTrigger("provision", AccessRights.Manage, Connection = "AzureWebJobsServiceBus")]string myQueueItem, TraceWriter log)
{
    System.Threading.Thread.Sleep(20000);
    log.Info($"C# ServiceBus queue trigger function processed message: {myQueueItem}");

    string EndpointUrl = "https://kk-db-governance-engine.documents.azure.com:443/";
    string PrimaryKey = "xx";
    var docClient = new DocumentClient(new Uri(EndpointUrl), PrimaryKey);
    var result = docClient.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri("kk-governance", "test-function-concurrency"), new Run());

}

Functions app.settings: enter image description here

If I add 10 messages to the queue, 10 documents are added to the database concurrently. I would expect that only 3 at the would be added with 20 secs delayed. Setting the WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT in the app settings does not seem seem to work. Any suggestions?

I also tried the maxConcurrentCalls in the host.json file:

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsServiceBus": "xxx",
    "AzureWebJobsStorage": "xx",
    "AzureWebJobsDashboard": "xx"
  },
  "version": "2.0",
  "extensions": {
    "serviceBus": {
      "maxConcurrentCalls": "3"
    }
  }
}

As you can see on screenshot below I use consumption plan: enter image description here

Answer: I got following working:

"serviceBus": { "maxConcurrentCalls": 3
}

1
c#azureazure-functionsazure-servicebus-queuesazure-webjobssdk
When you mean 10 functions, you mean you want to max a maximum of ten concurrent calls to SP ??? – Thomas
Yes exactly. I have one single function, and I want 10 concurrent instances of that single function running at all time. So if I add 10000 messages to the queue, 10 instances will run concurrently until all messages has been processed. – Thomas Segato
As per this post github.com/Azure/azure-functions-host/issues/…, you can set this setting WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT to 1. It will force having only one instance of your function app running. And then you can change the max concurrent call to 10 in the host.json file. I think both storage queues and servicebus queues should work fine. – Thomas
WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT should be app setting. just put this in the app settings in the portal – Thomas
Also if you want to configure the host.json file, check out the documentation: docs.microsoft.com/en-us/azure/azure-functions/… – Thomas

1 Answers

5
votes

An answer can be found here: Add Singleton support for Functions to ensure only one function running at a time

  1. If you're running on the consumption plan: To ensure you won't scale out to more than one instance, set this app setting:
    • WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT = 1

  2. To allow only X concurrent calls in this instance, you can specify this setting in the host.json file:

    { "version": "2.0", "extensions": { "serviceBus": { "maxConcurrentCalls": 1 } } }