Kafka topic partition zero is not getting used

Question

I created a topic with 3 partitions

Topic:fixtures  PartitionCount:3    ReplicationFactor:1 Configs:
    Topic: fixtures Partition: 0    Leader: 0   Replicas: 0 Isr: 0
    Topic: fixtures Partition: 1    Leader: 0   Replicas: 0 Isr: 0
    Topic: fixtures Partition: 2    Leader: 0   Replicas: 0 Isr: 0

I use a Java Producer to write into the topic synchronously

producer.send(new ProducerRecord<>(KafkaProperties.TOPIC_FIXTURES, key, value)).get();

I have a consumer in Java which subscribes and reads from it

Consumer<String, String> kafkaConsumer = new KafkaConsumer<>(configs);
kafkaConsumer.subscribe(Collections.singletonList(KafkaProperties.TOPIC_FIXTURES));

My keys are always a fixed set of 3 different strings (k1,k2,k3). But my messages are always going to either partition 1 or partition 2 -- k1 and k2 goes to partition 1 and k3 goes to partition 2.

Why is partition 0 unused?

Mickael Maison Mickael Maison · Accepted Answer · 2018-02-26T20:15:00

For keyed messages, the default partitioner computes the partition based on a hash of the key.

With many keys, using a hash of the key is an efficient method to spread messages evenly across partitions.

But if you only have very few different keys (or if the number of keys is close to the number of partitions), it's possible it won't use all the partitions.

If that's the case, then you can provide your own partitioner to the Producer using the partitioner.class configuration to do round-robin for example.

Kafka topic partition zero is not getting used

1 Answers