Spring cloud stream kafka metrics are not shown in actuator metrics | Springboot 2.2.2 | SpringCloud Hoxton.SR4

Question

I am not seeing the kafka consumer/producer metrics using actuator /actuator/metrics. Only basic jvm metrics is shown.

{
"names": [
"jvm.buffer.memory.used",
"jvm.threads.states",
"jvm.memory.committed",
"spring.integration.channels",
"process.uptime",
"jvm.memory.used",
"jvm.gc.pause",
"logback.events",
"http.server.requests",
"jvm.memory.max",
"tomcat.sessions.active.current",
"jvm.buffer.total.capacity",
"system.cpu.usage",
"jvm.threads.live",
"jvm.classes.unloaded",
"jvm.classes.loaded",
"jvm.threads.peak",
"tomcat.sessions.rejected",
"tomcat.sessions.alive.max",
"jvm.gc.memory.promoted",
"jvm.buffer.count",
"jvm.gc.memory.allocated",
"tomcat.sessions.expired",
"tomcat.sessions.created",
"jvm.gc.max.data.size",
"system.cpu.count",
"spring.integration.handlers",
"spring.integration.sources",
"process.start.time",
"jvm.threads.daemon",
"tomcat.sessions.active.max",
"jvm.gc.live.data.size",
"process.cpu.usage"
]
}

My application is working fine and binding looks ok too and visible in /actuator/bindings.

[
{
"bindingName": "send-kafka-out",
"name": "test.topic",
"group": null,
"pausable": false,
"state": "running",
"extendedInfo": {
"bindingDestination": "test.topic",
"ExtendedProducerProperties": {
"autoStartup": true,
"partitionCount": 1,
"headerMode": "headers",
"extension": {
"bufferSize": 16384,
"compressionType": "none",
"sync": false,
"sendTimeoutExpression": null,
"batchTimeout": 0,
"messageKeyExpression": null,
"headerPatterns": null,
"configuration": {},
"topic": {
"replicationFactor": null,
"replicasAssignments": {},
"properties": {}
},
"useTopicHeader": false,
"recordMetadataChannel": null
},
"validPartitionSelectorProperty": true,
"validPartitionKeyProperty": true
}
},
"input": false
}
]

Below is my stream configuration

public interface KafkaStreams {
    String OUTPUT = "send-kafka-out";

    @Output(OUTPUT)
    MessageChannel getOutboundMessageChannel();

}

@SpringBootApplication
@EnableDiscoveryClient
@EnableJms
@EnableBinding(KafkaStreams.class)
@ComponentScan("com.test.connector.*")
public class ConnectorApplication {

    public static void main(String[] args) {
        SpringApplication.run(ConnectorApplication.class, args);
    }

}

Below is my config.yml

spring:
  jms:
    listener:
      auto-startup: true
      acknowledge-mode: AUTO
      max-concurrency: 10
      concurrency: 10
  cloud:
    stream:
      default-binder: kafka-default
      function:
        definition: 
      kafka:
        binder:
          brokers:
            - localhost:9092
          configuration:
            default.key.serde: org.apache.kafka.common.serialization.Serdes$StringSerde
            default.value.serde: org.apache.kafka.common.serialization.Serdes$BytesSerde
            commit.interval.ms: 1000
            #security:
            #  protocol: SASL_PLAINTEXT

      bindings:
        send-kafka-out:
          destination: test.topic
          contentType: text/plain
          group: output-group-1
          binder: kafka-default
          producer:
            header-mode: headers

      binders:
        kafka-default:
         type: kafka
         environment:
           spring:
             cloud:
               stream:
                kafka:
                  binder:
                    brokers: localhost:9092

I am using SpringBoot 2.2.2.RELEASE, SpringCloud Hoxton.SR4 and spring-cloud-stream-binder-kafka 3.0.4.RELEASE. Any pointers about what am i missing would help.

@Gary Russell, Could you please help me with this. I am kinda stuck here. Even tried a new application with above versions and could reproduce it. — Anshul Bhatnagar

sobychacko sobychacko · Accepted Answer · 2020-07-06T14:52:20

You need to have input binding. The Kafka binder metrics only support consumers. It looks like you only have an output binding in the code. I just tried a quick app with a consumer in it and I was able to retrieve the metrics:

/actuator/metrics/spring.cloud.stream.binder.kafka.offset

{"name":"spring.cloud.stream.binder.kafka.offset","description":"Unconsumed messages for a particular group and topic","baseUnit":null,"measurements":[{"statistic":"VALUE","value":0.0}],"availableTags":[{"tag":"topic","values":["uppercase-in-0"]},{"tag":"group","values":["anonymous.06730d87-0871-4b82-bd82-ae4958dbe2a3"]}]}

Spring cloud stream kafka metrics are not shown in actuator metrics | Springboot 2.2.2 | SpringCloud Hoxton.SR4

1 Answers