Kafka rebalance minimisation

0
votes

in simple scenario where I have: 1 kafka broker N topics No topic partitioning

I want , for example, separate topics between consumers.

I load K consumers(streams), each stream to process N/K topics.

  1. I can use the same consumer group
  2. I can load each consumer in its own group

Can you confirm, that rebalance in 2-d case will not happen if new consumer added? This is good, because I would like avoid rebalance.

What are hidden disadvantages of both approaches?

1
consumerbalanceapache-kafka
if you have only one partition / topic then only one consumer from a group will be able to consume from it , if u add new consumer to the same group kafka will trigger a rebalance – user2720864

1 Answers

1
votes

Rebalance shouldnt happen when adding a consumer to a different group.

When using same consumer group, if one consumer dies, others will take his topics/partitions and continue processing from the offset it died. If you use different consumer groups you'll have to restart the consumer from the right place yourself.