Not able to consume messages using kafka-console-consumer.sh

Question

I have set up an AWS cluster of 3 nodes.I have modified the /etc/hosts file for the nodes which looks something like this

172.31.9.2 kafka1 172.31.9.2 zookeeper1 172.31.19.230 kafka2 172.31.19.230 zookeeper2 172.31.35.20 kafka3 172.31.35.20 zookeeper3

when I run the command from one of the nodes

bin/kafka-console-consumer.sh --bootstrap-server kafka1:9092 --topic first_topic --from-beginning

it works but when I replace the hostname by ip and run it with the below command

bin/kafka-console-consumer.sh --bootstrap-server 172.31.9.2:9092 --topic first_topic --from-beginning

it does not work. Do I need to change certain configs?

Where are you running kafka-console-consumer from? The nodes, or somewhere else (e.g. your local machine)? — Robin Moffatt
I am running from one of the nodes where I have hosted kafka — Nagaraj B

CantankerousBullMoose CantankerousBullMoose · Accepted Answer · 2018-06-20T19:00:21

When specifying the hostname in the 'listeners' configuration item in server.properties, use 0.0.0.0 as the host to make kafka accept connections more permissively. There's some documentation to that effect here I believe the recommended access method is to keep it locked down as you have it now and access brokers by host name, though.

Not able to consume messages using kafka-console-consumer.sh

1 Answers