org.apache.kafka.clients.NetworkClient Bootstrap broker (id: -3 rack: null) disconnected when connecting to kafka consumer via NiFi

1
votes

I am trying to consume messages from a kafka topic on Amazon MSK (Kafka Version 2.2.1) using NiFi ConsumeKafka_2_6 processor (NiFi version 1.12.1).In ConsumeKafka processor I am getting the warning as "org.apache.kafka.clients.NetworkClient [Consumer clientId=console-consumer-34567-5, groupId=nifi-consumer] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected".What can be the cause of this issue?Is this due to version mismatch of kafka client used or due to connectivity issues.

2
apache-kafkaapache-nifiaws-msk

2 Answers

0
votes

It's a connectivity problem or you configured the Nifi processor incorrectly

You're not connecting to MSK, you're trying to connect Nifi to itself

Bootstrap broker localhost:9092

Also, yes, you might have some issues using the 2_6 consumer with a 2.2 cluster, but very minor as the core functions should still work

0
votes

From aws cli we use the JVM truststore to talk to the MSK cluster. So from NiFi application also I was supposed to use that. I have configured the security protocol as SSL and configured JVM truststore in SSLContextService. Then I was able to consume messages from Kafka.