0
votes

When defininig a kafka topic, I can set the replication factor, and usually Kafka will distribute the partitions across the available brokers.

Say I have 3 brokers and have a topic with several partitions and a replication-factor=2, then some partitions will be set to be on brokers [0,1], some one [0,2] and some on [1,2] (ignoring the leader selection/preference for now)

Can I later eg set only one partition to have a replication factor of 3 (and thus add this partition to the remaining node) without having to also do this for the other partitions? Is there any drawback to having a number !=all of the partitions replicated to more/less nodes than the others?

All documentation etc I saw only ever speaks of the replication factor in terms of whole topics

I know I use kafka-reassign-partitions on a per-partition basis, but I found no discussion or recommendations on whether it is good/bad to only reassign a few partitions or what the dangers (if any) might be

1

1 Answers

0
votes

Only topics have replication factors, not partitions.

Partitions can become out-of-sync, and lose leadership in the case of errors, in which case, one or more partitions becomes unavailable, but the topic is still the source of how many replicas there ought to be

I found no discussion or recommendations on whether it is good/bad to only reassign a few partitions

You'd usually only do this for maintenance or cluster expansion to remove large partitions from occupying too much disk space on certain brokers.

The main "bad" thing would be how fast your network is to transfer the partition while still handling traffic to the other replicas of the topic.