Not able to understand correctly, how Event Hub consumer group working

1
votes

Requirement I want to process event data (same data) from multiple consumers parallel

enter image description here

What I understand from documentation to process same data form event hub we need to create multiple consumer groups.

Consumer groups enable multiple consuming applications to each have a separate view of the event stream, and to read the stream independently at its own pace and with its own offsets.

Message Retention 1, Partition count 3

enter image description here

Problem: I'm not getting data form log1 consumer group, Instead log1 When I tried $Default and log2 consumer group I was getting data in parallel.

Can anyone help me understand this problem?

1
azureazure-eventhub

1 Answers

1
votes

Based on the official doc , your understanding is correct. I tried to create my event hub as same as yours to reproduce your issue.

enter image description here

enter image description here

And I start 2 EPH via java code to monitor Log1 and log2 separately.

Then I send 10 messages to event hub to observe the 2 EPH console log.

enter image description here

It seems that I can receiver messages from both of them,however the order is messy. I guess the your issue is delay.

Update Answer:

Per my knowledge, delay in Azure event hub may be affected by many aspects. Maybe network , internal internal mechanisms. However ,Azure event hub ensure that your data is not lost during effective storage time.

If you need to ensure real time data , you could use Kafka which is used for building real-time data pipelines and streaming apps.