Azure Event Hub - Can't understand Java flow

Question

from Microsoft EventHub Java SDK examples (https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-java-get-started-send), these are the steps that needs to be taken to be able to consume messages from an Even-Hub via the java SDK :

1.Create a storage account

2.Create a new class called EventProcessorSample. Replace the placeholders with the values used when you created the event hub and storage account:

3.

  String consumerGroupName = "$Default";
   String namespaceName = "----NamespaceName----";
   String eventHubName = "----EventHubName----";
   String sasKeyName = "----SharedAccessSignatureKeyName----";
   String sasKey = "----SharedAccessSignatureKey----";
   String storageConnectionString = "----AzureStorageConnectionString----";
   String storageContainerName = "----StorageContainerName----";
   String hostNamePrefix = "----HostNamePrefix----";

   ConnectionStringBuilder eventHubConnectionString = new ConnectionStringBuilder()
        .setNamespaceName(namespaceName)
        .setEventHubName(eventHubName)
        .setSasKeyName(sasKeyName)
        .setSasKey(sasKey);

There are several things i don't understand about this flow -

A. Why is a storage account required? Why does it needs to be created only when creating a consumer and not when creating the event hub itself?

B. What is 'hostNamePrefix' and why is it required?

C. More of a generalaztion of A, but i am failing to understand why is this flow so complicated and needs so much configuration. Event Hub is the default and only way of exporting metrics/monitoring data from Azure which is a pretty straightforward flow - Azure -> Event Hub -> Java Application. Am i missing a simpler way or a simpler client option?

Anish K Anish K · Accepted Answer · 2019-09-19T21:09:44

All your questions are around consuming events from Event hub.

Why is a storage account required?

Read the event only once: Whenever your application will read event from event hub, you need to store the offset(identifier for the amount of event already read) value somewhere. The storing of this information is known as 'CheckPointing' and this information will be stored in Storage Account.
Read the events from starting everytime your app connects to it: In this case, your application will keep on reading the event from very beginning whenever it will start.

So, the storage account is required to store the offset value while consuming the events from event hub in case if you want to read event only once.

Why does it needs to be created only when creating a consumer and not when creating the event hub itself?

As it depends upon the scenario, whether you want to read your events only once or every time your app starts and that's why storage account is not required while creating event hub.

What is 'hostNamePrefix' and why is it required?

As the name states 'hostNamePrefix', this is name for your host. The host means the application which is consuming the events. And it's a good practice to make use of GUID as a hostNamePrefix. HostNamePrefix is required by the event hub to manage the connection with the host. In case, if you have 32 partitions, and you have deployed 4 instances of your same application then 8 partition each will be assigned to your 4 different instances and that's where the host name helps the event hub to manage the information about the connection of the respective partitions to their host.

I will suggest you to read this article on event hub for clear picture of the event processor host.

Azure Event Hub - Can't understand Java flow

1 Answers