Can't Start Kafka Server on Windows 10 - Kafka's log directories (and children) should only contain Kafka topic data

1
votes

Following instructions in lesson 28 of Learn Apache Kafka for Beginners Udemy course to start zookeeper and then start a kafka server broker on Windows 10. Zookeeper runs fine on port 2181:

C:\kafka_2.12-2.3.1> zookeeper-server-start.bat config/zookeeper.properties

...

INFO binding to port 0.0.0.0/0.0.0.0:2181

But after adding the bat files to path, running kafka server does not work:

C:\kafka_2.12-2.3.1> kafka-server-start.bat config/server.properties

...

ERROR There was an error in one of the threads during logs loading: org.apache.kafka.common.KafkaException: Found directory C:\kafka_2.12-2.3.1\data\kafka, 'kafka' is not in the form of topic-partition or topic-partition.uniqueId-delete (if marked for deletion). Kafka's log directories (and children) should only contain Kafka topic data. (kafka.log.LogManager)

Some of the stdout logging in zookeeper looks informative:

Accepted socket connection from /127.0.0.1:49439 (org.apache.zookeeper.server.NIOServerCnxnFactory) [2019-11-03 17:22:42,278] INFO Client attempting to establish new session at /127.0.0.1:49439 (org.apache.zookeeper.server.ZooKeeperServer) [2019-11-03 17:22:42,286] INFO Creating new log file: log.1 (org.apache.zookeeper.server.persistence.FileTxnLog)

...

INFO Processed session termination for sessionid: 0x1007b0044a40000 (org.apache.zookeeper.server.PrepRequestProcessor) [2019-11-03 17:22:42,987] INFO Closed socket connection for client /127.0.0.1:49439 which had sessionid 0x1007b0044a40000 (org.apache.zookeeper.server.NIOServerCnxn)

Under the data folder I created, there are two folders I created, the second of which was filled after I tried running the kafka broker:

kafka/ |-- empty zookeeper/ |-- version-2/ |--log.1

Why does this error happen and how can I start a Kafka server on Windows 10?

EDIT:

Contents of config/server.properties:

num.network.threads=3

num.io.threads=8

socket.send.buffer.bytes=102400

socket.receive.buffer.bytes=102400

socket.request.max.bytes=104857600

log.dirs=C:/kafka_2.12-2.3.1/data/

num.partitions=1

num.recovery.threads.per.data.dir=1

offsets.topic.replication.factor=1

transaction.state.log.replication.factor=1

transaction.state.log.min.isr=1

log.retention.hours=168

log.segment.bytes=1073741824

log.retention.check.interval.ms=300000

zookeeper.connect=localhost:2181

zookeeper.connection.timeout.ms=6000

group.initial.rebalance.delay.ms=0

1
apache-kafkawindows-10
Could you please post the content of config/server.properties (comments can be excluded) ? – iamabug
Sorry, could you also please show the directory names and filenames under C:\kafka_2.12-2.3.1\data\kafka ? – iamabug
@iamabug I edited the question with the file contents and server.properties. – Wassadamo
I think @cricket_007 is right. The error message says Kafka's log directories (and children) should only contain Kafka topic data. Removing zookeeper directory should be the solution. – iamabug

1 Answers

0
votes

The data/ folder contains empty zookeeper/ and kafka/ directories

Did you create those folders? Kafka's log.dirs can only contain folders in the format of topic-partition or topic-partition.uniqueId-delete (if marked for deletion), as mentioned by the error. That folder should be empty (no subdriectories) if starting fresh installation.

Plus, Kafka data directory should not hold Zookeeper data as Zookeeper should be independent of Kafka