Azure WebJob Output List of Messages to Queue via ICollector seems slow

Question

I have an Azure WebJob that loops through the pages of a file and processes them. The job also has an ICollector to an output queue:

[Queue("batch-pages-to-process")] ICollector<QueueMessageBatchPage> outputQueueMessage

I need to wait until all of the pages are processed before I send everything to the output queue, so instead of adding each message to the ICollector in my file processing loop, I add the messages to a list of queue messages:

List<QueueMessageBatchPage>

After all of the pages have been dealt with, I then loop through the list and add the messages to the ICollector:

foreach (var m in outputMessages)
{
    outputQueueMessage.Add(m);
}

But this last part seems to take a long time. To add 300 queue messages, it takes almost 50 seconds. I don't have much to gauge by, but that seems slow. Is this normal?

Answer 1

There's no objective standard of slow vs. fast to offer you, but a few thoughts:

a) Part of the queuing time will be serialization of each QueueMessageBatchPage instance... the performance of that will be inversely related to the breadth and depth of the object graphs those instances represent. More data obviously takes more time to write to the queue.

b) I know you mentioned that you can't write to the queue until all file lines have been processed, but if at all possible you might reconsider that choice. To the extent you could parallelize both the processing of lines in the file and subsequent writing to the output queue (using either multiple WebJob instances or perhaps TPL Tasks within a single WebJob instance), you could potentially get this work done a lot faster. Again, I realize you stated upfront that you can't do that, so I'm just suggesting you consider the full implications of that choice (if you haven't already).

c) One other possibility to look at... make sure the region where your storage queue lives is the same as where your WebJob lives, to minimize latency.

Best of luck!