kafka Reassignment of partition is still in progress

Question

I have a kafka cluster 21,22,23,24,25, however some brokers was permanently removed,

remaining 23,24,25, then the __consumer_offsets topic partition lose leader:

Topic: __consumer_offsets  Partition: 0  Leader: 24 Replicas: 24 Isr: 24
Topic: __consumer_offsets  Partition: 1  Leader: -1 Replicas: 21 Isr: 21

partition 1 lose leader, and when I run reassign it to 25, it shows still processing, always.

And info like this：

Topic: __consumer_offsets  Partition: 1  Leader: -1 Replicas: 25,21 Isr: 21

How could I reassign partition 1 to available broker, without syncing from removed broker?

reassign json partial file:

   {
        "version":1,
        "partitions":[
            {
                "topic":"__consumer_offsets",
                "partition":1,
                "replicas":[
                    25
                ],
                "log_dirs":[
                    "any"
                ]
            }
        ]
    }

kafka version: kafka_2.12-1.1.0

OneCricketeer OneCricketeer · Accepted Answer · 2020-12-16T15:09:57

Your offsets topic appears to only have had one replica. It's therefore not possible to recover that partition's data as it's waiting for a broker ID of 21 to become available

The only "fix" without creating a new broker with that id is by manually modifying the Zookeeper data to change the replica assignment

Once you do fix it, you should increase the replica count of that topic as no consumers will work if it's unavailable

kafka Reassignment of partition is still in progress

1 Answers