Stackdriver : create alert policy based on simple arithmetic involving two user-defined metrics

Question

I have two user-defined metrics: totalCount and failedCount. I've combined them into a single stacked-bar graph on a Stackdriver dashboard. All looking good so far.

I'd like to alert if failedCount exceeds 20% of the totalCount, but I can't find a way to achieve this. The dialogue to create an alert policy only allows me to work with one metric, but I need both in order to perform the simple arithmetic.

What am I missing?

Answer 1

What you want to do is not supported at the moment in google not such Datadog, there are other option such like When creating Alert Policy you have the option to add more than one metric in one Alert see below an example and use just the policy Triggers, see below:

You can see check this stack for more details on the arithmetic operation.

Answer 2

I think that the ability to create the kinds of alerting you'll be talking about will be included in Service Monitoring (see the Service Level Objectives section):

https://cloud.google.com/service-monitoring/

Answer 3

You can create exactly the alerting policy you're describing using a ratio alert. Unfortunately, you can't create those through the UI, you have to use the API (https://cloud.google.com/monitoring/api/ref_v3/rest/v3/projects.alertPolicies#metricthreshold - check out "denominatorFilter").

The gcloud CLI tool does support everything the API does, so you don't actually have to write code if you don't want to - see https://cloud.google.com/monitoring/alerts/using-alerting-api#api-create-policy.

Aaron Sher, Stackdriver engineer