Kafka expectantly shutting down. License topic could not be created

3
votes

On Kafka startup multiple messages are logged to kafka/logs/kafkaServer.out and contain:

INFO [Admin Manager on Broker 0]: Error processing create topic request CreatableTopic(name='_confluent-license', numPartitions=1, replicationFactor=3, assignments=[], configs=[CreateableTopicConfig(name='cleanup.policy', value='compact'), CreateableTopicConfig(name='min.insync.replicas', value='2')]) (kafka.server.AdminManager)

After approx 15 minutes Kafka shuts down and outputted to kafka/logs/kafkaServer.out is :

org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1.
[2020-12-08 04:04:15,951] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown

(kafka.server.KafkaServer) org.apache.kafka.common.errors.TimeoutException: License topic could not be created Caused by: org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1. [2020-12-08 04:04:15,952] INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer)

It appears Kafka shuts down because the replication factor is set to 3 for the topic _confluent-license ? I'm not creating the topic _confluent-license, is this created as part of Kafka startup for licensing check ?

In attempt to fix I've modified /v5.5.0/etc/kafka/server.properties so that replication factor is 1 for internal topics:

############################# Internal Topic Settings  #############################
# The replication factor for the group metadata internal topics "__consumer_offsets" and "__transaction_state"
# For anything other than development testing, a value greater than 1 is recommended for to ensure availability such as 3.
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1

instead of 3 :

#offsets.topic.replication.factor=3
#transaction.state.log.replication.factor=3

But this does not fix the issue and same logs are generated. The replication factor of __consumer_offsets is still 3. How to reduce the replication factor of topic _confluent-license from 3 to 1 ? Or could there be an alternative issue that is causing Kafka to shutdown ?

3
apache-kafkaconfluent-platform
Note that the second error says Error checking or creating metrics topic, which has its own setting separate from the license topic. I've rolled back your question because that isn't part of the answers - OneCricketeer

3 Answers

2
votes

You should change the property confluent.license.topic.replication.factor by default it is 3.

2
votes

(kafka.server.KafkaServer) org.apache.kafka.common.errors.TimeoutException: License topic could not be created Caused by: org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1. [2020-12-08 04:04:15,952] INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer)

The above error is due to the license topic having a default replication factor as 3. The same can be configured with confluent.license.topic.replication.factor to be equal to 1 if you have only 1 broker. The documentation for the same is here.

[2020-12-08 07:46:02,241] ERROR Error checking or creating metrics topic (io.confluent.metrics.reporter.ConfluentMetricsReporter) org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 2 larger than available brokers: 1.

The above error is due to the Confluent Metrics Reporter being enabled. The replication factor for the metric topic is defaulted at 3 and can be configured with confluent.metrics.reporter.topic.replicas to be 1 if you have just one broker. The documentation for the same is here.

-2
votes

The property confluent.license.topic.replication.factor is not working as expected.

Instead, we can try anyone of these properties confluent.license.replication.factor=1 or confluent.license.admin.replication.factor=1 if the number of broker is 1. This property is not in the documentation.