Kafka, Re-Partitions, locality and ordering

Question

I have a question about re-partitioning in Kafka...

Let say I have a Topic which has 3 partitions on 3 Broker Kafka Cluster with replication factor 3, which will mean every Broker will have one partition for master and 2 other brokers will have the replication of this partition and I know inside of a partition, as long partition distribution strategy stay same, ordered delivery inside of the partition guaranteed.

My question is, lets say, I decide 3 partitions are not enough, I decide to add one more partition and one more broker, what will happen then? Would the message that are already in a partition would be transferred to new partition if the partition strategy say so or they will stay in the old partition and only the new records would be written to new partition?

If the all the partition will be re-arranged after the creation of new partition, will the order of delivery be preserved?

Answer 1

Be aware that one use case for partitions is to semantically partition data, and adding partitions doesn't change the partitioning of existing data so this may disturb consumers if they rely on that partition. That is if data is partitioned by hash(key) % number_of_partitions then this partitioning will potentially be shuffled by adding partitions but Kafka will not attempt to automatically redistribute data in any way.

https://kafka.apache.org/documentation/#basic_ops_modify_topic