Unable to calculate the ratio of two metrics in Prometheus

0
votes

I am collecting ZFS metrics in Prometheus v.2.19 from Linux servers using ZFS Exporter. The data is being collected from all targets and the values are correct, however there is this strange issue: I want to calculate the percentage of the ratio of ARC misses vs. ARC hits, so I use the following formula:

100 * rate(zfs_arc_stats{stat='misses'}[5m]) / rate(zfs_arc_stats{stat='hits'}[5m])

but it produces no data. I can get the values from each of the rate expressions individually, but not from the above formula. If I use the formula with the same labels, ie.:

100 * rate(zfs_arc_stats{stat='hits'}[5m]) / rate(zfs_arc_stats{stat='hits'}[5m])

it gives the correct results of 100, because the ratio of misses to misses or hits to hits is always 1:1. I tried similar formulas with metrics from other sources (ie. PCP, Collectd) and those work fine, ie. I could calculate ratios of same metrics with different labels, so it seems like the problem could be specific to the metrics from ZFS Exporter. This exporter has a very few metrics, but each metric has many different stats identified by the value of the 'stat' label. All the metrics are of gauge type, but I do not think it really matters for the rate function in PromQL. Anyway like I mentioned before the rates can be computed separately, only the ratio fails. Somebody please suggest how to tackle this issue.

1
prometheusmetricspromql
I was actually after the hit percentage where the formula is: 100 * rate(zfs_arc_stats{stat='hits'}[5m]) / ignoring(stat) (rate(zfs_arc_stats{stat='hits'}[5m]) + ignoring(stat) rate(zfs_arc_stats{stat='misses'}[5m])) – mac13k
A commit was made in github.com/mac133k/zfs_exporter to address this issue, so now the code in the master branch contains a new metric namespace without excessive labeling. The old metric naming is still available in the labelhell branch. – mac13k

1 Answers

2
votes

Prometheus divides one metric by another only if all the labels with their values for both metrics are identical. In this case metrics on the left side of / have label {stat='misses'}, while metrics on the right side have different value for stat label: {stat='hits'}. Prometheus provides ignoring and on operations, which may be applied to any binary operator (i.e. /, +, -, etc.). See the corresponding docs for details.

So in you case you must tell Prometheus to ignore stat label when performing calculations:

100 * rate(zfs_arc_stats{stat='misses'}[5m]) / ignoring(stat) rate(zfs_arc_stats{stat='hits'}[5m])