Azure iot hub & azure function: consumer group issue

Question

I have iot hub in azure and azure function, which is triggered when a message from a device is sent. The function looks like this:

[FunctionName("Ingest")]
public async Task Run(
            [IoTHubTrigger("IoTHubName", Connection = "IoTHubConnectionString"

#if DEBUG
            , ConsumerGroup = "localdev"
#else
            , ConsumerGroup = "release"
#endif

            )] EventData[] messages)
{
   ...
}

IoTHubName and IoTHubConnectionString I'm getting from local.settings.json when developing locally and from env variables in azure (Configuration tab in the portal) in the release. Almost all the time it's working fine.

But sometimes function is not triggered. I eliminated, that it's somehow connected with the consumer group. Because when I change "release" to something new, the function starts to get triggered again. So, my main question: what can be an issue? How can I find if some other function listens to the same consumer group(which is likely not possible, but who knows...)? Or maybe how can I remove all consumers? In fact how to understand why my function sometimes is not triggered?

When an Azure function connects to an IoT hub it silently uses an epoch value. This essentially forces a consumer group to only have one listener. The one with the highest epoch value wins. is it possible that you have something else connecting to the release consumer group with a high epoch value that could be causing the function to fail to connect? You should always provide an Azure function with a dedicated consumer group. — Mark Radbourne

iAviator iAviator · Accepted Answer · 2021-02-10T08:33:34

Provide Azure function with a dedicated consumer group. Refer architecture diagram below which is documented by Azure here.https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-features#consumer-groups

In the picture above, Azure function is one of the event listener and it has a dedicated consumer group. As per Azure In a stream processing architecture, each downstream application equates to a consumer group. In your case downstream application is Azure function. Also, if you have multiple consumers i.e. multiple Azure function listening to a consumer group then each consumer is listening to a particular partition i.e Azure funtion1 listens to partition 1 and Azure funtion2 listens to partition2. This will guarantee that the message is delivered to one of the consumer and not all of them.

Azure iot hub & azure function: consumer group issue

1 Answers