0
votes

Reading Event Hub documentation and creating a simple producer-consumer example

Link -> https://docs.microsoft.com/en-us/javascript/api/overview/azure/event-hubs-readme?view=azure-node-latest

I was wondering in a production application how this would work. The reason is that in the current implementation is listening for a specific amount of time then the connection is closing.

Should we send the request to specific REST endpoints and activate the listeners after the producer finishes?

1

1 Answers

3
votes

You are correct that in most production scenario's this does not work. Best is to keep the listener open during the lifetime of the application. In most cases when a restart of the application is triggered, processing should resume from the last checkpoint on continuation. The example does not cover this.

From the docs:

For the majority of production scenarios, we recommend that you use the event processor client for reading and processing events. The processor client is intended to provide a robust experience for processing events across all partitions of an event hub in a performant and fault tolerant manner while providing a means to checkpoint its progress. Event processor clients can work cooperatively within the context of a consumer group for a given event hub. Clients will automatically manage distribution and balancing of work as instances become available or unavailable for the group.

Here is an example of processing events combined with checkpointing. For demo purposes the listener stops after a while. You will have to modify the code to run as long as the process is not stopped.

Checkpointing is important if you have a continuous flow of events being send. If the listener is not available for some period you do want to resume processing not from the beginning of the first event nor from new events only. Instead you will want to start from the last know processed event.