What is the command to list down all the available brokers in Apache Kafka?

Question

I want to run a multi node cluster in Apache Kafka. I have made three server.properties files - server, server1 and server2. I have also given then different broker ids and different port numbers. Still on running the script kafka-topics.sh with a replication factor if 3 ,it throws an error saying that the replication factor:3 is larger than the number of brokers:0

I used this command :

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic my-replica-topic

Error shown is

Error while executing topic command replication factor: 3 larger than available brokers: 0 kafka.admin.AdminOperationException: replication factor: 3 larger than available brokers: 0 at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70) at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:171) at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93) at kafka.admin.TopicCommand$.main(TopicCommand.scala:55) at kafka.admin.TopicCommand.main(TopicCommand.scala)

Could you let me know where I am going wrong ?

What different commands did you use to start up your three different kafka servers? — Morgan Kenyon
The commands were pretty straight-forward : bin/kafka-server-start.sh config/server.properties , bin/kafka-server-start.sh config/server-1.properties and bin/kafka-server-start.sh config/server-2.properties . All these servers had different broker ids and ports. — Afan
I know the commands to start them as well, but trying to help you debug your problem the exact commands you ran could be helpful. — Morgan Kenyon
I ran these exact commands in sequence . I think there was some error in server.properties file .I got a fresh copy of server.properties installed, and then tried to run again the 3 servers following the same set of commands. It worked then. Anyway , thank you for the help. :) — Afan
You're welcome, glad to help. Could you answer your question by adding some information about what you did to solve your problem? So if someone finds this question later they can see what you did to solve your issue. — Morgan Kenyon

Kevin Li Kevin Li · Accepted Answer · 2016-08-08T01:04:00

I think u should at least start 3 kafka server to make sure the number of brokers is larger or equal to the number of replication factor:

First we make a config file for each of the brokers:

cp config/server.properties config/server-1.properties

cp config/server.properties config/server-2.properties

Now edit these new files and set the following properties:

config/server-1.properties:
    broker.id=1
    listeners=PLAINTEXT://:9093
    log.dir=/tmp/kafka-logs-1


config/server-2.properties:
    broker.id=2
    listeners=PLAINTEXT://:9094
    log.dir=/tmp/kafka-logs-2

The broker.id property is the unique and permanent name of each node in the cluster. We have to override the port and log directory only because we are running these all on the same machine and we want to keep the brokers from all trying to register on the same port or overwrite each others data. We already have Zookeeper and our single node started, so we just need to start the two new nodes:

bin/kafka-server-start.sh config/server-1.properties

bin/kafka-server-start.sh config/server-2.properties

Now create a new topic with a replication factor of three:

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic my-replicated-topic

What is the command to list down all the available brokers in Apache Kafka?

1 Answers