It depends on Group ID. Suppose you have a topic with 12 partitions. If you have 2 Kafka consumers with the same Group Id, they will both read 6 partitions, meaning they will read different set of partitions = different set of messages. If you have 4 Kafka cosnumers with the same Group Id, each of them will all read three different partitions etc.
But when you set different Group Id, the situation changes. If you have two Kafka consumers with different Group Id they will read all 12 partitions without any interference between each other. Meaning both consumers will read the exact same set of messages independently. If you have four Kafka consumers with different Group Id they will all read all partitions etc.
topics
are not queues, because once a message is consumed from atopic
, it stays there(unless its lifetime has expired) and theoffset
moves to the next, whereas for a queue, once a message is consumed, the message is removed from that queue. Ordered sets is also bypartitions
only. – jumping_monkey