how to force delete topic from kafka

2
votes

we do the following steps in order to delete the topic - hgpo.llo.prmt.processed

but even after 12 hours topics folders still not deleted from /var/kafka/kafka-logs

note - we set - delete.topic.enable=true

  hkafka01 kafka-logs]# /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper master01:2181 --alter --topic hgpo.llo.prmt.processed--config retention.ms=1000

  WARNING: Altering topic configuration from this script has been deprecated and may be removed in future releases. Going forward, please use kafka-configs.sh for this functionality Updated config for topic "hgpo.llo.prmt.processedd"

  kafka01 kafka-logs]# /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper master01:2181 --delete --topic hgpo.llo.prmt.processed 

  Topic hgpo.llo.prmt.processed is already marked for deletion.

how to force the topic deletion ?

on kafka machine ( production machine ) , we see that: ( no free space )

 /dev/sdb    15500450528 15500434144         0 100% /var/kafka

and under /var/kafka/kafka-logs , we still see all topic folders (huge size) as example: ( down )

so is it possible just to remove these folders as example - by

rm -rf hgpo.llo.prmt.processed-28 ?

under /var/kafka/kafka-logs we have many topic folders as :

 117G hgpo.llo.prmt.processed-28 
 117G hgpo.llo.prmt.processed-29
 117G hgpo.llo.prmt.processed-3
 117G hgpo.llo.prmt.processed-30
 117G hgpo.llo.prmt.processed-31
 117G hgpo.llo.prmt.processed-32

. .

2
apache-kafka

2 Answers

4
votes

Deletion the way you've been trying should work but I'm guessing Kafka is not working well with a full disk.

As your disk is full, I'm assuming Kafka is not usable and a short downtime is acceptable. To delete the topic manually:

  • Shutdown Kafka
  • On all Kafka brokers that have replicas for hgpo.llo.prmt.processed, go to their data directory and delete the folders starting with hgpo.llo.prmt.processed-.
  • Connect to Zookeeper with the zookeeper-shell tool
  • Run: rmr /brokers/topics/hgpo.llo.prmt.processed
  • Restart Kafka

To prevent this from happening again, I suggest you take a look at the retention policies Kafka offers to control disk usage

2
votes

A Kafka topic marked for deletion can be deleted by directly logging into the ZooKeeper shell and removing the topic metadata from the cluster. We also recommend removing the log segments from each broker that are associated with the topic.

  1. zookeeper-shell
  2. rmr /config/topics/
  3. rmr /brokers/topics/
  4. rmr /admin/delete_topics/
  5. For each broker:
    • Shut down the broker
    • rm -R ${log.dirs}/-*
    • Start up the broker

Note: Step 5 should be done with care to ensure you are only removing the log directories for the topic you are intending to delete