Prometheus: getting cumulative KwH from milliwatt samples

Question

I have an energy monitoring device, a Belkin Wemo, that reports current usage in milliwatts, and I have those being exported into prometheus. What I'd like is a graph that shows me cumulative KwH, monotonically increasing, since the moment I started collecting data.

The following query plotted in a table with a min_step=1h shows me KwH for each hour, and they add up to what I believe the total KwH is, so I'm pretty sure my data is correct:

sum_over_time(current_power[1h])/1000/1000/count_over_time(current_power[1h])

Plotting that same query in a graph does not do what I want, because I want a cumulative total, not a series of individual hourly totals. If I could just sum the results of this query, I think it would do what I want. However, the sum() operator just doesn't seem to do what I think it would when wrapped around the above query.

Answer 1

I think I came up with a workable solution, using Prometheus subqueries. I believe I had to upgrade my version of Prometheus to get the subquery support. With subqueries, it's possible to compute the Kilowatt-Hours and then sum them up. I started with this:

sum_over_time( (sum_over_time(current_power[1h])/1000/1000/count_over_time(current_power[1h]))[1y:1h] )

Then I wanted higher resolution (the above will lag a bit), so I reduced the intervals down to 5 minutes:

sum_over_time( (sum_over_time(current_power[5m])/1000/1000/count_over_time(current_power[5m]))[1y:5m] )/12

This gives me KwH for the last year. I've only tested it for a day and a half.

Answer 2

sum_over_time sums up the metric within the timeframe you define for it (the bit in the square brackets) getting a commulative number from a gauge / histogram is technically impossible unless you are going to keep your data around forever, which is technically impossible :) to properly get what you want, you are going to need to turn this from a gauge / histogram metric into an increment metric