Whether kafka broker holds replication set instead of partitions?

1
votes

From the Kafka documentation and from some other blogs I read, I concluded that one kafka-broker consists of one partition of topics. Here It says one Kafka-broker holds only one partition. I have only one broker in my system, But I can able to create a topic with 3 partitions and 1 replication factor. I also tried to create a topic with 3 partitions and 3 replication factor with only one broker. It throws below error

Error while executing topic command : replication factor: 3 larger than available brokers: 1 [2017-10-21 15:35:25,928] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: replication factor: 3 larger than available brokers: 1 (kafka.admin.TopicCommand$).

So I have a query.

  1. Whether Kafka-broker holds replication instead of a partition?
  2. If I create 3 partitions with a single broker, what happens?
  3. In such case of 1 broker, 1 replica and 3 partition , how many partitions of single topic can kafka-broker hold?

Somebody, please explain what happens here.

2
apache-kafka

2 Answers

4
votes

The post you are reffering to doesn't say that one broker can store only one partitions. It just says that partition is not splittable between brokers (topic is). Actually I manage a brokers with thousands of partitions. So, for your questions:

  1. Kafka brokers hold many partitions. Replication is the way to store multiple copies of partitions across cluster.
  2. If you create topic with 3 partitions on single node cluster, the broker will hold the data for all the partitions. Replication is not possible since it requires more nodes.
  3. All of them.
1
votes

Summary: The replication factor has to be equal or less in number compared to the number of brokers you have.