Not able to delete Kafka topic from cmd utility and programmatically (Java)

Question

I am not able to delete the Kafka topics, they are just marked for deletion. What's the issue? Am I missing something? Any suggestions are highly welcomed.

I tried to delete Kafka topic using both ways:

First, I ran the command:

kafka-topics.bat --delete --zookeeper localhost:2181 --topic Cat

Second, I tried programmatically

public void deleteSomeTopic() throws Exception {

  consumer = new SimpleConsumer(host, port, soTimeout,  bufferSize,"deleteClient");

  List<String> listTopics = new ArrayList<>();
  TopicMetadataRequest request = new TopicMetadataRequest(listTopics);

  //consumer will send request and get corresponding response
  TopicMetadataResponse response = consumer.send(request);

  //getting topicMetadata list
  List<TopicMetadata> topicMetadataList = response.topicsMetadata();

  for (TopicMetadata topicMetadata: topicMetadataList) {
     System.out.println("Do you want to delete this Topic: "+ topicMetadata.topic());
     String ch = scanner.next();
     if(ch.equalsIgnoreCase("y"))
        AdminUtils.deleteTopic(zkClient, topicMetadata.topic());
     Thread.sleep(2000);
  }
}

Both executes successfully, but when I listed topics using command

kafka-topics.bat --list -zookeeper localhost:2181

It gives output looks like

Ape - marked for deletion   
Cat - marked for deletion     
Dog - marked for deletion   
Elephant                    
apple                       
carrot                      
ginger - marked for deletion
guava                       
mango

My server.properties file also has

controlled.shutdown.enable=true
delete.topic.enable=true

I am using kafka_2.10-0.8.2.2 and my pom.xml file has only two dependencies.

<dependencies>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming_2.10</artifactId>
        <version>1.5.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming-kafka_2.10</artifactId>
        <version>1.5.1</version>
    </dependency>
</dependencies>

I also checked tmp/kafka-logs directory, topics directories are still there. Any workaround I need to do?

Morgan Kenyon Morgan Kenyon · Accepted Answer · 2015-11-17T14:39:22

Based on the output of your list command, you seem to be correctly telling Kafka you want to delete a topic. For some reason Kafka just isn't deleting it. There was a bug in Kafka prior to version 0.8.2 which prevented topic deletion. Looking at the comments of that bug request, there still seems to be some issues with 0.8.2.2, which is what you're running. So maybe this bug hasn't been totally fixed?

Based on your kafka.bat file, are running Kafka on a windows environment? Have you made many changes to the default server.properties file? Or are you basically using the default file that comes with Kafka? This could be a Windows related issue, I've only used it on Linux. Using almost all the default values, only setting delete.topic.enable=true deletion has always worked for me on Linux. Random guess, but maybe this wasn't fixed on Windows environment or in .bat file?

Though a work around seems to exist based on this Stack Overflow answer and the comments on the original bug report. I have never used it though, maybe give it a shot?

Not able to delete Kafka topic from cmd utility and programmatically (Java)

1 Answers