How to send message to a particular partition in Kafka?

Question

I have created a topic that has many partitions. Using the console producer I want to send messages to particular partitions and view the through the console consumer. On the console producer I have tried this,

kafka-console-producer.bat --broker-list localhost:9092 --topic sample  --property parse.key=true --property key.separator=,

Send messages as,

key1,another-message

But I am just confused on whether key1 represents partition number.

Using the console consumer I viewed the messages,

kafka-console-consumer.bat --zookeeper localhost:2181 --topic sample

I want to view the messages according to the partitions. Is this the right way to view the messages on the console consumer? Can anyone please provide a clear understanding on this?

Note that if you are not using the console producer but a Kafka producer client (like the official Java producer client) you can actually directly specify the target partition that a message should be sent to. The console producer is nice for playing around but typically not used in production. — Michael G. Noll
Possible duplicate of How to produce messages to selected partition using kafka-console-producer? — Jacek Laskowski

Katya Gorshkova Katya Gorshkova · Accepted Answer · 2018-05-14T07:04:05

The key is not the partition number but Kafka uses the key to specify the target partition. The default strategy is to choose a partition based on a hash of the key or use round-robin algorithm if the key is null.

If you need a custom algorithm to map the messages to partitions you need to implement org.apache.kafka.clients.producer.Partitioner interface. The name of you class must be set as a partitioner.class property of the producer.

Update: You can also specify partition number directly in the ProducerRecord

How to send message to a particular partition in Kafka?

1 Answers