How to configure server.properties for clustering in Kafka

0
votes

I've been following Kafka Quickstart for "Setting up a multi-broker cluster" on a single machine. (Just for testing purposes).

Running Kafka with three properties files worked good. (I ran them on a single machine for testing)

server.properties : 

broker.id=0
listeners=PLAINTEXT://:9092

server-1.properties :

broker.id=1
listeners=PLAINTEXT://:9093

server-2.properties :

broker.id=2
listeners=PLAINTEXT://:9094

Now, I want to create a cluster with three machines.

1) Do I run three Zookeeper for three machines? With the same port (2181)? Or Run just one Zookeeper on one machine?

2) When I run Kafka with server.properties, I know that I should have different broker.id for each machine. How about the listeners part? Do I use the same port?

listeners=PLAINTEXT://192.168.0.5:9092 (machine 1)
listeners=PLAINTEXT://192.168.0.6:9092 (machine 2)
listeners=PLAINTEXT://192.168.0.7:9092 (machine 3)

1
apache-kafka

1 Answers

2
votes
  1. The number of Zookeeper machines affects service availability and reliability. For testing purpose, one is enough. If three machines, using same port or different ports are both ok because there is a conf in server.properties:
zookeeper.connect=localhost:2181
# if using three zookeeper machines and different ports, modify it to following
# zookeeper.connect=192.168.0.5:2181,192.168.0.6:2182,192.168.0.7:2183
  1. Same port is good and recommended. Also make sure to set advertised.listeners to an address that is resolvable by each of the machines in the clusters as well as from where your clients will run.