Monitoring Timer Triggered Azure Functions For Inactivity

Question

We have a couple of functions in a function app. Two of them are triggered by a timer, do some processing and write to queues to trigger other functions.

They normally work very well until recently where the timer trigger just stopped triggering. We fixed this by restarting the application which resolved the issue. The problem is that we were completely unaware of the trigger stopping as there were no failures and the function app is not constantly 'looked at' by our people.

I'd like to configure automatic monitoring and alerting for this special case. I configured Application Insights for the function app and tried to write an alert which watches the count metric of the functions which are triggered by a timer. If the metric is below the set threshold (below 1 in the last 5 minutes) the alert should be triggered.

I tested this by just stopping the function app. My reasoning behind this was that a function app that does not run should fullfill this condition and should trigger an alert within a reasonable time frame. Unfortunately this was not the case. Apparently a non-existing count is not measured and the alert will never be triggered.

Did someone else experience a similar problem and has a way to work around this?

What is the timer interval and approx how long the timer function run? — Shahid Syed
I have not experienced this before but have you thought about using Azure Event Grids to monitor the function? docs.microsoft.com/en-us/azure/event-grid/overview — M0rty
Ideally your function applications wouldn't stop triggering. This is likely an issue with your triggers synchronizing. Consider filing an issue on the Functions GitHub repo. — Connor McMahon
The timer triggers every minute and runs for a couple of seconds (less than 10), @ShahidSyed. — WalternativE
I've not tried to use EventGrid before but I'll look into it, thank you for your input @M0rty. — WalternativE

Valdemar_Rudolfovich Valdemar_Rudolfovich · Accepted Answer · 2019-01-14T11:20:42

I've added Application Insights alert:

Type: Custom log search
Search query: 
  requests | where cloud_RoleName =~ '<FUNCTION_APP_NAME_HERE>' and name == '<FUNCTION_NAME_HEER>'
Alert logic: Number of results less than 1
Evaluated based on: Over last N hours, Run every M hours

Alert fires if there are no launches over last N hours.

Monitoring Timer Triggered Azure Functions For Inactivity

1 Answers