Azure Event Hub - Partitions usecase question

0
votes

I'm new to Azure Event Hubs and I'm having a hard time understanding the Partitions.

I have the following scenario:

  • 1 Event Hub Namespace

  • 1 actual Event Hub

  • 2 Partitions in the Event Hub

  • 2 Consumer groups

  • 1 Event Producer

  • 2 Event Consumers, one per Consumer group

The Event Producer sends out 10 events to the Event hub. The events gets distributed to the partitions with a round-robin mechanism. So the Event hub looks like this:

Partition 1: [0] [2] [5] [6] [8]
Partition 2: [1] [3] [4] [7] [9]

When the Event Consumers start reading, each consumer would end up with only a part of the events, like so:

Consumer 1: Gets events 0,2,5,6,8
Consumer 2: Gets events 1,3,4,7,9

Is it true that a Consumer group can only access a subset of the Partitions?

My assumption is that the Event Hub architecture supports broadcasting of events to multiple consumers. And that every consumer wants all the events. But it seems to me that Event Hub isn't designed to have all consumers get all the events, but I don't understand why that would be useful.

Can anyone help me understand Partitions?

1
azureazure-eventhub

1 Answers

2
votes

Each Event Hubs partition is a persistent stream of events that is available to all consumers, regardless of which consumer group that are associated with. Any consumer can read from any partition at any point in the event stream.

Partitions are used to help scale resources to support a greater degree of concurrency and increase throughput for the Event Hub. Generally speaking, the more partitions that are in use, the more concurrent operations the Event Hub can handle. More information can be found in the Event Hubs overview.

My assumption is that the Event Hub architecture supports broadcasting of events to multiple consumers.

Not quite; consumers are responsible for pulling events from the partitions of an Event Hub, they are not pushed to consumers. Any consumer with permissions can connect to a partition and read independently. Events are not removed once read, they exist in the partition until their age exceeds the retention period.

But it seems to me that Event Hub isn't designed to have all consumers get all the events

That is not correct. Event Hubs exposes the events for any consumer wishing to read them. Using a client like the EventProcessorClient from the Event Hubs SDK allows an application to consume from all partitions without having to manage each partition consumer individually.