Azure Monitor alert is sending false failure email notifications of failure count of 1 for Functions app

0
votes

I have a Functions app where I've configured signal logic to send me an alert whenever a failure greater than or equal to one has occurred in my application. I have been getting emails everyday saying my Azure Monitor alert was triggered followed by an email later saying that the failure was resolved. I know that my app didn't fail because I checked in Application Insights. For instance, I did not have a failure today, but did have a failures the prior 2 days: Azure Function Failures

However, I did receive a failure email today. If I go to configure the signal logic where I set a static threshold of failure count greater than or equal to 1 it shows this: failure count

Why is it showing a failure for today, when I know that isn't true from the Application Insights logs? Also, if I change the signal logic to look at total failures instead of count of failures, it looks correct:
enter image description here

I've decided to use the total failures metric instead, but it seems that the count functionality is broken.

Edit: Additional screenshot: Configure signal logic

1
azureazure-functionsazure-application-insights
May I know which signal did you choose for the alert ? And could you please share a screenshot of the "Configure signal logic" page ? – Hury Shen
Hey @HuryShen, I chose the "Failures" signal. The second and third image are both from the "Configure signal logic" page, but I just added a screenshot of the entire page. – ddx
I think it is difficult for us to analyze the cause of the problem based on the available information. You can raise a support ticket on azure portal by following the steps on this page. Azure support team will help you on it. – Hury Shen
@JohnT, what's the type of your function? time trigger or other triggers? – Ivan Yang
@IvanYang it's a Timer Trigger – ddx

1 Answers

1
votes

I suggest you can use Custom log search as the signal if you have already connected your function app with Application insights(I'd like to use this kind of signal, and don't see such behavior like yours).

The steps as below:

Step 1: For signal, please select Custom log search. The screenshot is as below:

enter image description here

Step 2: When the azure function times out, it will throw an error and the error type is Microsoft.Azure.WebJobs.Host.FunctionTimeoutException, so you can use the query below to check if it times out or not:

exceptions 
| where type == "Microsoft.Azure.WebJobs.Host.FunctionTimeoutException"

Put the above query in the "Search query" field, and configure other settings as per your need. The screenshot is as below:

enter image description here

Then configure other settings like action group etc. Please let me know if you still have such issue.

One thing should be noted: Some kinds of triggers support retry logic, like blogtrigger. So if it reties, you can also receive the alert email. But you can disable the retry logic as per this doc.