Azure Queue trigger - execute one message at the time not working

Question

I have an azure queue trigger associated to a queue, and I want to ensure that the trigger only reads and executes one message at the time. So, when the message is executed (successfuly or not) it processes the next message.

What is happening is that the queue executes one message, yet it begins to execute other message. My host.json:

"queues": {
    "maxPollingInterval": 20000,
    "visibilityTimeout": "00:01:00",
    "batchSize": 1,
    "maxDequeueCount": 5,
    "newBatchThreshold": 1
  }

Following instructions from MS link:

https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue#trigger---configuration

If you want to avoid parallel execution for messages received on one queue, you can set batchSize to 1

So it would be expected to run only one message at the time (I'm using consumption plan).

This is critical because I need to ensure that only one message it processed at the time.

Is there any setting that I could change?

Or is queue trigger not a good option to address this requirement?

Answer 1

Storage Queue does NOT guarantee ordering - so if needing sequential processing because order of delivery matters you need to consider Azure Service Bus and set in Function setting (host.json)

maxConcurrentCalls = 1

Even if you do the trick by setting maximum number of instances that a function app can scale to as follow, ordering is still not guaranteed with Azure Storage Queue.

WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT = 1

Microsoft documentation is not the perfect one. It's being continuously updated.

Answer 2

If you want to minimize parallel execution for queue-triggered functions in a function app, you can set the batch size to 1. But this setting eliminates concurrency only so long as your function app runs on a single virtual machine (VM).

If you have multiple Virtual Machines and function instances on each VM, there will be one message processed for each function instance running in each virtual machine.

This microsoft document explains concurrency on triggers.

Answer 3

For those coming across this question looking to debug locally and getting issue with multiple queue items making this difficult, you can add the following to your local.settings.json to override the default functionality on your machine only:

{
  "IsEncrypted": false,
  "Values": {
     "AzureFunctionsJobHost__extensions__queues__batchSize": 1
   }
}

Documentation