Kafka ISR wrongly update

1
votes

I'm facing some problems related to kafka. I created one topic with 2 partitions and 1 replica to test the kafka behavior.

kafka_2.10-0.9.0.0/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 2 --topic t1

Retrieving the topics related data I got this:

kafka_2.10-0.9.0.0/bin/kafka-topics.sh --describe --topic t1 --zookeeper localhost:2181
Topic:t1        PartitionCount:2        ReplicationFactor:1     Configs:
        Topic: t1       Partition: 0    Leader: 0       Replicas: 0     Isr: 0
        Topic: t1       Partition: 1    Leader: 1       Replicas: 1     Isr: 1

If I turn off broker 0 sometimes the topics related data remain the same even if from the log of broker 1 I can see the string "successfully elected as leader", while other times they change as follow:

kafka_2.10-0.9.0.0/bin/kafka-topics.sh --describe --topic t1 --zookeeper 
Topic:t1        PartitionCount:2        ReplicationFactor:1     Configs:
        Topic: t1       Partition: 0    Leader: -1        Replicas: 0     Isr: 
        Topic: t1       Partition: 1    Leader: 1       Replicas: 1     Isr: 1

For sure the first behavior is wrong (as the ISR 0 does not exists). What should be the expected behavior? If the second is the right one, is it true to say that from that moment all the sent messages will end on Partition 1?

In both cases analyzing the zookeper fs with 

ls /brokers/ids

it shows me only the Broker 1

2
apache-kafka

2 Answers

1
votes

You created the topic with 1 as the replication factor. That means each partition only exists on 1 broker.

So in your example, if you take down broker 0, t1-0 becomes offline, its leader is -1 (none) and it has no Isr. At this stage no clients can use it to send or receive message. So the behaviour you pasted above is expected.

The first behaviour you describe is incorrect. As you've not pasted any related output it's hard to tell what happened.

If you want partitions to stay available when you take down 1 broker, you need to increase the replication factor (to 2 in your example).

0
votes

Highlighting what Mickael said, the replication factor isn't the number of the copies that your partition will have other than the leader one; the replication factor is the total number of replicas that the partition will have including the leader.

So the behaviour you copied here is right. Each partition has just one replica (the leader one) and if the related broker goes down, the partition is totally offline so no leader (-1) no ISR.