3
votes

I have multiple Prometheus instances providing the same metric, such as:

my_metric{app="foo", state="active",   instance="server-1"}  20
my_metric{app="foo", state="inactive", instance="server-1"}  30
my_metric{app="foo", state="active",   instance="server-2"}  20
my_metric{app="foo", state="inactive", instance="server-2"}  30

Now I want to display this metric in a Grafana singlestat widget. When I use the following query...

sum(my_metric{app="foo", state="active"})

...it, of course, sums up all values and returns 40. So I tell Prometheus to sum it by instance...

sum(my_metric{app="foo", state="active"}) by (instance)

...which results in a "Multiple Series Error" in Grafana. Is there a way to tell Prometheus/Grafana to only use the first of the results?

2

2 Answers

1
votes

I don't know of a distinct, but I think this would work too:

topk(1, my_metric{app="foo", state="active"} by (instance))

Check out the second to last example in here: https://prometheus.io/docs/prometheus/latest/querying/examples/

0
votes

One way I just found is to additionally do an average over all values:

avg(sum(my_metric{app="foo", state="active"}) by(instance))