Kafka topic is getting reappeared after 10 sec of deletion

3
votes

I'm facing issue with kafka topic deletion.

Using kafka rest API's to create/delete topic and for producing & consuming messages. I have tried to delete the topic where the topic will be deleted, but after some time say 10 sec, topic gets reappeared.

Have checked the Consumer Group offsets and LAG is listed as negative.

docker run --net=host --rm confluentinc/cp-kafka:3.1.0 kafka-consumer-groups --zookeeper localhost:2181 --describe --group grp1

GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG OWNER

grp1 topic1 0 3 0 -3 none

2
apache-kafkakafka-consumer-api

2 Answers

3
votes

It seems deleting a kafka topic is still has some bugs.

* The only way to delete a topic permanently is as follows: *

  • stop the brokers

  • remove the directories on disk

sudo rm -rf kafka_data_dir/topic_name

  • remove the topic from zookeeper:

bin/zkCli.sh - to start zookeeper shell

rmr /config/topics/topic_name

rmr /brokers/topics/topic_name

rmr /admin/delete_topics/topic_name

2
votes

You might have enabled the auto.create.topics.enable property which automatically creates a topic if any Producer [or] Consumer issues a request to Kafka broker with send / subscribe / assign request.

Once you have deleted the topic, either Consumer / Producer issued a request to the broker which in-turn created a new topic (with the same name).

You can disable the property(default: true) and re-test your setup. Use AdminUtils to create the topic. Topic deletion is much improved in the latest version of Kafka.