Error: Replication factor: 1 larger than available brokers: 0, when I create a Kafka topic

Question

I have one problem in running Kafka in the cluster. I explain it one by one. First, When I run Kafka commands on the cluster CSSH from my computer, I get this error:

Error while executing topic command : Replication factor: 2 larger than available brokers: 1. [2019-01-06 15:12:36,587] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 2 larger than available brokers: 1. (kafka.admin.TopicCommand$)

In fact, I run CSSH on my computer to access to the cluster, after running Zookeeper and Kafka server on the cluster, when I run command of creating a topic I get the error. In the cluster, I have these setting on the server.properties on node1:

broker.id=1
port=9092
listeners=PLAINTEXT://150.20.11.137:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
sockeet.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=10073741824
log.retention.check.interval.ms=300000
zookeeper.connect= 150.20.11.134:2186, 150.20.11.137:2186, 
150.20.11.157:2186
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0

In zookeeper.properties of each node I have this setting:

 dataDir=/tmp/zookeeper
 clientPort=2186
 maxClientCnxns=0

Also, I run each command on the cluster nodes to run Kafka and Zookeeper:

./bin/zookeeper-server-start.sh ./config/zookeeper.properties
./bin/kafka-server-start.sh ./config/server.properties

After that, I want to create a topic with this command in the cluster and then I get above error on each node:

./bin/kafka-topics.sh --create --zookeeper localhost:2186 -- 
 replication-factor 2 --partitions 3 --topic testFlink

Would you please tell what the problem is exactly? And what is wrong in my cluster setting?

Thanks in advance.

Answer 1

I doubt that you've been able to form a cluster successfully. While booting up the cluster, please ensure you're starting all the 3 zookeeper nodes first, and then the three brokers. You can refer to this post, to check if Kafka has formed a cluster.

Update: I had overlooked the zookeeper properties you're using, which is missing the essential key-value pairs, required for creating a cluster. The following properties for zookeeper should be good to start with. Considering you have 3 zookeeper nodes, the zookeeper.properties or zoo.cfg (if it's a standalone zk) file on the zk nodes should look something like the following.

zk-1 properties

tickTime=2000
initLimit=10
syncLimit=5
clientPort=2186
dataDir=/opt/zookeeper/data
server.1=0.0.0.0:2888:3888
server.2=<zk_2-ip>:2888:3888
server.3=<zk_3-ip>:2888:3888

zk-2 properties

tickTime=2000
initLimit=10
syncLimit=5
clientPort=2186
dataDir=/opt/zookeeper/data
server.1=<zk_1-ip>:2888:3888
server.2=0.0.0.0:2888:3888
server.3=<zk_3-ip>:2888:3888

zk-3 properties

tickTime=2000
initLimit=10
syncLimit=5
clientPort=2186
dataDir=/opt/zookeeper/data
server.1=<zk_1-ip>:2888:3888
server.2=<zk_2-ip>:2888:3888
server.3=0.0.0.0:2888:3888

Okay, so before you start the zookeeper processes, you need to do one more thing. Checkout the dataDir property that you're using, in this example it's /opt/zookeeper/data. For each of the zookeepers, you've to create a file called myid and enter the value of 1 for zk-1, 2 for zk-2 and 3 for zk-3. Then you start the zookeepers, and it should form a cluster. You can use a bash cmd like echo "1" > /opt/zookeeper/data/myid for zk-1. Rest will be similar.

Answer 2

The only reason this error could occur is if your brokers are not connected to zookeeper cluster.
How are you running your Kafka broker? Are you running it as a service or a daemon process? Can you share logs?

You can check if a broker is running simply by running the command

nc -vz 150.20.11.137 9092

This would fail if the broker is not running.

I would suggest you use a zookeeper WEB UI like Zookeeper Navigator. Running it on docker is the easiest way. Here's the docker compose file for it. This will help you identify if there's any problem with zookeeper and all brokers are indeed connected to it.

I would also suggest you use yahoo Kafka manager. Its a web UI for your cluster and the most elegant way to manage your cluster. Here's docker for it.

And yes, Your zookeeper properties need to include the server IPs of all other zookeeper nodes like explained by Bitswazsky.