Scaling Long Running Message Processing Azure Service Bus

Question

What is the best way to scale a Worker Role that is processing many long running Azure Service Bus messages using the QueueClient Message Pump.

If using QueueClient.OnMessageOptions.MaxConcurrentCalls = 6 and QueueClient.OnMessage does that mean i can only process a max of 6 messages at a time?

Is it bad form to have the long running processing within the OnMessage callback to spawn a new Task to complete it's processing?

Should i be using the QueueClient.OnMessageAsync instead?

Thanks for any help.

Answer 1

By “long running” do you mean IO-bound or CPU-bound?

Assuming IO-bound then I wouldn’t spawn a new Task in the OnMessage callback. This creates thread management overhead that can slow processing down at scale.

Consider using OnMessageAsync if you are using IO-bound operations and make sure that you await the asynchronous implementations of any of these operations. This uses your existing threads much more efficiently.

If your operations are CPU-bound then Task creation may do more for you. The mechanics of this are discussed in a series of excellent posts by Stephen Cleary:

http://blog.stephencleary.com/2013/10/taskrun-etiquette-and-proper-usage.html

The MaxConcurrentCalls property controls the number of concurrent requests to the service bus. Increasing this number has a limited impact if you’re IO-bound and limited by available bandwidth. I would recommend doing a bit of performance testing with the Azure client-side performance counters to get the optimum value for your environment.