Will two Azure Event Hub consumers get the same notifications?

Question

I'm developing an Azure IoT web application that consumes events from an Azure Event Hub. I regularly publish the web application to Azure in a testing environment and then I also run the it in debug mode from my own machine and when that happens, essentially there are two web applications running at the same time that are consumers of the Event Hub.

I'm wondering if one consumer might "swallow" a notification so that the other consumer will not get it. That would be bad and I would have to set up a separate Event Hub just for my debugging local machine.

Will both consumers of the Event Hub get all the notifications?

I haven't set up the Event Hub or the consumer code in any special way. The consumer code is using an Azure.Messaging.EventHubs.EventProcessorClient and its ProcessEventAsync() looks like this:

async Task ProcessEventHandler(ProcessEventArgs eventArgs)
{
    string deviceId = eventArgs.Data.Properties["deviceId"].ToString();
    string message = Encoding.UTF8.GetString(eventArgs.Data.Body.Span);
    await ProxyMessageToController(deviceId, message);

    
    await eventArgs.UpdateCheckpointAsync(eventArgs.CancellationToken);
}

Jesse Squire Jesse Squire · Accepted Answer · 2020-08-06T21:46:20

The EventHubProcessorClient will ensure that there is only one active reader for a given partition within the scope of the consumer group that you specified when creating the processor. So long as each EventHubProcessorClient reading the Event Hub uses the same consumer group, they will not both actively read from the same partition.

What is important to know is that EventProcessorClient coordinates with other processors to distribute load between them. When one processor "steals" ownership of a partition from other processors, you will potentially see some overlap where both have read the same events. This overlap is generally a very small window of time but does depend somewhat on your checkpointing strategy. You're most likely to see partition ownership change when starting a new processor until all partitions are equally distributed.

The other caveat is that the Event Hubs service has an at-least-once delivery guarantee. It is possible, though rare, for events to be duplicated though they were published a single time.

Will two Azure Event Hub consumers get the same notifications?

1 Answers