Spring Boot Kafka - Message management with consumer different

0
votes

My application create with SpringBoot and is in cluster (two different istance openshit) Every istance has one consumer that read message of topic in replication factory.

I would like to find a mechanism to block the reading of a message into topic in replication factory if it has already been read by one of the two consumers

Example:

CONSUMER CLIENT A -- READ MSG_1 --> BROKER_1 - Offset increase - Commit OK

CONSUMER CLIENT B --> NOT READ MSG_1 --> BROKER_1 -- Correct beacause already commit

Now BROKER_1 is show and new lead is BROKER_2

How can I block the already read message into BROKER_2?

Thanks all!

Giuseppe.

1
spring-bootapache-kafkakafka-consumer-api
if both consumers are on the same consumer group, none of your consumers will read msg_1 again, no matter the leading broker – aran

1 Answers

1
votes

Replication factor doesn't control if/how consumers read messages. The partition count does. If the topic only has one partition, then only one consumer instance is able to read messages, and all other instances are "blocked". And if the message is already read and commited then it doesn't matter which broker is the leader because the offsets are maintained per topic, not per replica

If you have more than one partition and you still want to block consumers from being able to read data, then you'll need to implement some external, coordinated lock via Zookeeper, for example