How to avoid race conditions using QueueTrigger in Azure web job?

Question

I've a WCF service which pushes the incoming SOAP message from the Third Party to the Azure Storage Queue.

The Third Party system is an automated system which fires the SOAP request messages in BULK. As soon the message is received in WCF, the WCF service reads the message and pushes it to the Azure Storage Queue, which then is read by an Azure Web Job configured using [QueueTrigger] attribute.

public async Task UpdateRatesAsync([QueueTrigger("queue-name")] string message, TextWriter log)

The role of this function is update the Rates of the Product in the system. The web job is triggered as soon as there's a message in the queue and it starts processing the request. When multiple rate messages for the same product are pushed, the QueueTrigger function executes which tries to update the Rates for the same Product at the same time resulting in a race condition.

How can I resolve this race condition? Is it by reading the message from the queue one by one?

Do I need to try a different operation to read the message from the queue by removing the QueueTrigger?

Don't forget to consider the situation where a job fails and gets placed back on the queue. This could also see you running an earlier (failed) job after a later (completed) job. — Llama
@John yes you're right, he can prevent that by setting the max dequeue count to 0 config.Queues.MaxDequeueCount = 0;, so if the job fails the message gets placed in a different "poison" queue, which can be handled differently. — A.J Alhorr

A.J Alhorr A.J Alhorr · Accepted Answer · 2019-06-03T06:55:55

If I remember correctly the QueueTrigger batch size defaults to 1 message at a time, of course you can edit the configurations of it manually, but you shouldn't be running into this problem in the first place unless you edited the batch size already.

static void Main()
{

    JobHostConfiguration config = new JobHostConfiguration();
    config.Queues.BatchSize = 1;
    var host = new JobHost(config);
    host.RunAndBlock();
    [...]

}

And yes, if the messages in the queues need to be executed in a certain order you should execute them one at a time, given that the execution of these messages is asynchronous.

How to avoid race conditions using QueueTrigger in Azure web job?

1 Answers