Azure Functions: how to efficiently send batch of messages to Service Bus using bindings?

Question

I'm working with Azure Functions 2.0 and trying to send a collection of messages to Azure Service Bus Topic. Currently I'm using IAsyncCollector<T> as output binding :

[FunctionName("MyFunction")]
public static async Task MyFunction([ServiceBus(MyTopicName, EntityType.Topic, Connection = ServiceBusConnection) outTopic])
{
    var messages = await GetMessages();
    foreach(var message in messages)
    {
        await outTopic.AddAsync(message);
    }
}

Such approach is pretty convenient (declarative code, no boilterplate) but brings one significant issue: for large number of messages the performance is unacceptable. Is there any other recommended way to work with large batches that need to be sent to Azure Service Bus ?

Not sure what optimizations you have tried, but there are some best practices here; docs.microsoft.com/en-us/azure/service-bus-messaging/… — Matt Evans
@MatthewEvans thank you for your response, I'm aware of that but ideally I would like to send messages in a batch using Azure Function bindings — mickl

Matt Evans Matt Evans · Accepted Answer · 2019-07-15T09:11:40

It looks like there is a known issue in using IAsyncCollector with ServiceBus (and other services.

https://github.com/Azure/azure-webjobs-sdk/issues/921

See first response re ordering and this :

This is easy when if we only send a small number of rows via IAsyncCollector because we could just buffer in -memory and force an ordering. But it’s prohibitive if we send huge payloads.

Azure Functions: how to efficiently send batch of messages to Service Bus using bindings?

1 Answers