Showing HTTP Request API latency using the Spring Boot Micrometer metrics

2
votes

We use Prometheus to scrape Spring Boot 2.0.0 metrics and then persist them in InfluxDB. We then use Grafana to visualize them from InfluxDB.

Our micrometer dependencies are

  • micrometer-core
  • micrometer-registry-prometheus

I want to be able to show a latency metric for our REST APIs.

From our Prometheus scraper I can see these metrics are generated for HTTP requests.

  • http_server_requests_seconds_count
  • http_server_requests_seconds_sum
  • http_server_requests_seconds_max

I understand from the micrometer documentation, https://micrometer.io/docs/concepts#_client_side, that latency can be done by combining 2 of the above generated metrics: totalTime / count.

However our data source is InfluxDB which does not support combining measurements, https://docs.influxdata.com/influxdb/v1.7/troubleshooting/frequently-asked-questions/#how-do-i-query-data-across-measurements, so I am unable to implement that function in InfluxDB.

Do I need to provide my own implementation of this latency metric in the Spring Boot component or is their an easier way that I can achieve this?

1
spring-bootinfluxdbprometheusmicrometerspring-micrometer
If you collect data with Prometheus - why don't you store & process it with Prometheus? Vice versa, if you intend to store & process your data in Influx - why don't you send it to Influx directly? – Yuri G
It is the common pattern we use for gathering metrics from microservices in our cluster. We use prometheus to scrape the metrics from services and export the data directly to influxdb. We could implement as you describe but then would not be following our common pattern. – Donal Hurley
I get the fact you use it that way right from your initial text, there was no need to repeat it. I was asking - WHY did you do it? See, in fact, you'd unnecessarily overengineered your solution (and to add insult to injury - you're trying to combine products with two significantly different approaches) - and that's the only how & why you ever stumbled upon a problem like this. – Yuri G

1 Answers

0
votes

You essentially can join your measurements in Kapacitor, another component of Influxdata TICK stack.

It's going to be pretty simple with JoinNode, possibly followed by Eval to calculate what you want right in place. There's tons of examples around it in documentation.

Although the problem is different there: you'd unnecessarily overingeneered your solution, and moreover - you're trying to combine two products that has the same purpose, but uses different approach to it. How smart is that?

You're already scraping things with Prometheus? Fine! Stay with it, do the math there, it's simple. And Grafana works with Prometheus too, right out of the box!

You wanna have your data in Influx (I can understand that, it's certainly more advanced)? Fine! Micrometer can send it right to Influx out of the box - and in at least two ways!

I, personally, don't see any reason to do what you suppose to do, can you share one?