Grafana state timeline panel with values (states) supplied by label

Question

The bounty expires in 2 days. Answers to this question are eligible for a +50 reputation bounty. forresthopkinsa wants to draw more attention to this question:

Need a way to use Prometheus labels as State Timeline values

I do have a Prometheus time series with samples like these:

a_metric{band="1", state="A"} 1
a_metric{band="2", state="C"} 1
a_metric{band="1", state="A"} 1
a_metric{band="2", state="C"} 1
a_metric{band="1", state="B"} 1
a_metric{band="1", state="B"} 1
...

I would like to visualize this time series in a state timeline panel such that bands become horizontal bands and states become discrete states within these bands. For this, I would have to extract values from the label state (and use them instead of values 1).

Is this possible and can such a visualization be achieved?

If I understand correctly Prometheus' label_values() cannot serve here, because it is restricted to templating. I suspect Grafana transformations could play a role, but I do not yet have experience with those. The complication also arises because Prometheus does not have string type metrics.

Answer 1

I don't think it is possible since as you said prometheus is not designed to use strings as values. If you have control over the exporter that is giving you these metrics, you can change the states to different values without the state label, and then use Value Mappings to map: 1 == State A; 2 == State B; etc

Surfacing it this way also will remove the issue of a given band being in multiple states at a given time. Storing the states as labels means your app needs to turn off a given label/state in addition to turning on the next state. If you forget to do this, then the metric will show the band in 2 states at once.

For the blackbox exporter, we get metrics like below with the query probe_success. This can only be 1/0 but the example would work if you have more states as well.

probe_success{instance="https://app1.example.com", job="blackbox"}  0
probe_success{instance="https://app2.example.com", job="blackbox"}  0
probe_success{instance="https://app3.example.com", job="blackbox"}  1
probe_success{instance="https://app4.example.com", job="blackbox"}  1

Set the Value Mappings to map from the number to string