0
votes

Say, there is a Consumer group. (Consumers with the same group ID).

The Consumer group is consuming Topic A from a Broker.

Topic A has 4 partitions, and there are 4 Consumers in that group.

Each Consumer consumes different partition. ( Consumer 1 takes messages in partition 1, Consumer 2 takes messages in partition 2 and so on because that's what consumer group does in kafka. Among Consumer Group, each has 1/4 of the topic.

My question : How do they share the message so that they all have Topic A?

How do they combine those bits and pieces? and where does this take place?

If my computer (consumer 1 of group A) consumes Topic A from a Broker, and my friend's computer (consumer 2 of group A) consumes other pieces of the same Topic, how do we combine the message in Topic A?

1

1 Answers

0
votes

I thought of the term 'Consumer' a computer or a server consuming a Topic from a Broker. That's why I got confused with Consumer group.

Consumer is a client or a program, and I can have many consumer's on my computer or server. Consumer Group means multiple consumer processes on an independent machine

So I don't need to worry about Consumer's in a group sharing bits of message to complete a Topic. Previously, I thought each consumer being a server or a computing resource, so they had to communicate somehow. But that's how I got confused. They don't need to communicate to each other over the network or need a pool to share their consumed partitioned.

Consumer 1 can read from partition 1, Consumer 2 can read from partition 2, and if Consumer 1, 2 share the same group ID (Consumer group), Consumer 1 doesn't need to read from partition 2, and Consumer 2 doesn't need to read from partion 1. They already have a Topic they need. Boom!

I posted an answer to help someone who thought like me.