Azure Function miss IoT trigger

1
votes

I am using the Azure function to process message from IOT hub and output to Blob storage. enter image description here

But the function missed IOT messages when I send in high frequency. For example, I send 30 messages from 20:40:16 to 20:40:23 but only 3 are processed and stored in to the Blob storage and I have no ideal where the rest 27 went. enter image description here

I am using the function consumption plan and Azure declares it will auto scaling depends on the load. But from the above activity log, it only one thread is running and not even queue the input and cause some message lost.

So, what I should do to catch all messages from IOT hub?

2
azureazure-functionsazure-iot-hub
Without seeing your code it is difficult to determine if the messages are really being lost or you have bug. Do you know that you can just set up a routing rule in the IoT hub and have it automatically write all device telemetry to blob storage? – Mark Radbourne
I am sure the messages are reached to the Azure since I can see them in ServiceBusExplorer. It may impossible to do the routing rule in my case since I send binary data that cannot in the JSON format to the Azure that is also why I need the function to decode my message than save to Blob – doushicai
Does this happened again ? or is this the only first time you have observed this behavior ? – SatishBoddu-MSFT
Hi SatishBoddu, in my observation, this happens all the time when the IoT message receive frequency higher than the Function process speed. This looks to me like the function is always single thread and never buffer the input. Is this what consumption plan should be or my setting problem? – doushicai

2 Answers

1
votes

Found the solution myself. The trigger need to change from Azure Event Hubs to Event Grid Trigger as the images show below.

Azure Event Hubs Azure Grid Trigger

0
votes

Azure Functions on a consumption plan can handle this load, but you might want to make a separate Consumer Group in your IoT Hub that the Function can use. In the Azure Portal, go to Built-in endpoints and add a new Consumer Group. enter image description here

You then have to specify in your Function which Consumer Group to use

    [FunctionName("Function1")]
    public static async Task Run([IoTHubTrigger("messages/events",ConsumerGroup = "functions", Connection = "EventHubConnectionAppSetting")]EventData message,

I tested this with a consumption plan Function listening to IoT Hub default endpoint and writing to blob storage with an 8 second delay to make it more like your function. I'm seeing no loss in messages, whether I send 30 or 100 messages. Make sure that no other applications are using your new Consumer Group!