Uusing Spring Kafka org.springframework.kafka.listener.ConcurrentMessageListenerContainer creates multiple listeners based on ContainerProperties, and the number of partitions in the topic. And the javadoc says "Messages from within the same partition will be processed sequentially". So if there is only 1 partition and concurrency is set to say, 10, what will happen - there won't be any concurrency? or will the messages be distributed to 10 listeners but out of order?
1
votes
After some more research, I guess if the there are 10 consumers with sample group id and the topic has 1 partition, only 1 out of the 10 consumers will get the messages and rest of the consumers will be idle. Is my understanding correct?
– user3366706
That is correct; only one consumer instance can consume from a single partition. Concurrency is limited by the number of partitions.
– Gary Russell
1 Answers
3
votes
No, there definitely will be only one target listener. One partition - one process to consumer per consumer group. That is an Apache Kafka nature. That is not Spring Kafka problem.
You can parallel messages from that partition lately from your listener method using TaskExecutor. But that is already your application - the Framework won't do anything for you on the matter. Just because the nature of the target Kafka system.
