Stackdriver policy does not resolved automatically

Question

i've added new alert policy on stackdriver in our project to be notified when GKE upgrade is in progress. i saw there are log entries that indicate such upgrade operation so i've created a log based metric (counter) for this long entries and created an alert then will fire the soon as this metric is above a Treshold of 0. (Configured with 'any time series violates' with condition is above treshold of 0 for most recent value).

when upgrade started i do get an alert that it started as expected. the thing is , when the upgrade is finished and this metric reach back to 0 , there is no 'resolved' operation on this policy.

why? what am i missing?

thanks

Answer 1

As it mentions in this documentation:

If an incident (upgrade in this case) is open, then the policy's set of conditions is currently being met or there is no data to indicate that the condition is no longer met.

Regardless of that, the issue will be marked as 'resolved' when no data is received for 7 days (after the upgrade is completed).