CloudWatch alarm always at insufficient data state

3
votes

I will want to get notified whenever there is a 500 error in my service. So, I created a metric filter in CloudWatch under a specific log group. Pattern is correct as per my application log logic. Metric is chosen as 1. Alarm is configured too with parameter values as below -

  • Period 1 min
  • Stat standard sum
  • whenever my metric filter is >=1 for 1 consecutive periods
  • whenever state is alarm is chosen under notification
  • remaining are straight forward

I manually called APIs of my application and generated 500 many time over many consecutive minutes. I have also used apache bench to generate such errors in bulk for consecutive minutes.

But always I do see that my alarm is in insufficient data. As a results, email notification which is configured as action to alarm does not get called.

How to get rid of the problem? Any suggestion will be helpful.

1
amazon-web-servicesamazon-cloudwatchamazon-cloudwatchlogs
It sounds like your CloudWatch Logs filter isn't working, can you verify that for us? Easy test: In AWS Console when you add a new filter there is a UI to test the filter against existing streams in your log group; do that, and if there are no results then your filter isn't working. – Anthony Neace
Insufficient data is the state your alarm will be in most of the time, when there are no instances of the metric filter being logged. If it isn't going into alarm state when the string in your metric filter is logged then you have the metric filter configured incorrectly. – Mark B
@anthony-neace: I already checked such test result, that is finding the content. Example output: Found 1 matches out of 50 event(s) in the sample log. – Martin
@mark-b: My metric filter pattern is "FAIL | 500". It matches in test result. Metric value is 1. What else is part of metric filter configuration? Alarm configurations are also already specified. – Martin

1 Answers

1
votes

I got the same issue as well.

INSUFFICIENT_DATA now got fixed with missing data in cloudwatch. Refer to cloudwatch console or api's on what to do when metric data is missing.

The only error I have seen is posting data to the right metric name and dimension. And the value of it.

If the value is always zero or undefined, the alarm will never get triggered.