0
votes

I've configured Spring boot actuator and micrometer for exposing Kafka consumer metrics. Some of the Kafka metrics don't have 'topic' label. For example,

kafka_consumer_records_consumed_total_records_total{client_id="consumer-1",topic="topic1",} 0.0
kafka_consumer_records_consumed_total_records_total{client_id="consumer-2",topic="topic2",} 0.0
kafka_consumer_records_consumed_total_records_total{client_id="consumer-3",topic="topic3",} 0.0

kafka_consumer_commit_latency_max_seconds{client_id="consumer-1",} 0.013
kafka_consumer_commit_latency_max_seconds{client_id="consumer-2",} 0.013
kafka_consumer_commit_latency_max_seconds{client_id="consumer-3",} 0.013

My question is

  1. Is there any configuration required to show 'topic' label for metric kafka_consumer_commit_latency_max_seconds?

  2. I noticed that client_id="consumer-1" is same for topic="topic1" even after application restart. Can I rely on client_id to be always point to the specific topic?

1

1 Answers

0
votes

Some metrics are global for the Consumer; some are specific to a topic and some are specific to a topic/partition.

The client.id is fixed; when generated by the kafka-clients, there is no guarantee you will always get the same one for a topic, but you can set a specific ConsumerConfig.CLIENT_ID for each consumer.