Kafka connect throws ClassNotFoundException when plugin.path has comma seperated values

0
votes

I am using Kafka connect to create a MQTT Kafka connection.I put all the kafka MQTT connector specific jar downloaded from confluent site to "/data" folder. And accordingly update the "connect-standalone.properties" file to reflect the plugin path i.e 

plugin.path=/opt/kafka_2.11-2.1.1/libs,/data

When I run the Kafka Connect 

./connect-standalone.sh ../config/connect-standalone.properties ../config/connect-mqtt-source.properties

I get following error :

[2019-07-18 10:26:05,823] INFO Loading plugin from: /data/kafka-connect-mqtt-1.2.1.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:220) [2019-07-18 10:26:05,829] ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectStandalone:128) java.lang.NoClassDefFoundError: com/github/jcustenborder/kafka/connect/utils/VersionUtil at io.confluent.connect.mqtt.MqttSourceConnector.version(MqttSourceConnector.java:29) at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:344)

screenshot as below : enter image description here

Please note that "connect-utils-0.3.140.jar" is present in "/data" folder as highlighted by red underlines.

Now If I make a soft link screenshot below or copy all the jars from "/data" folder and update the plugin path to :

plugin.path=/opt/kafka_2.11-2.1.1/libs

enter image description here

Kafka connect works perfectly fine.

Any help why it does not work in the first scenario i.e kafka connector specific jars in different folders

1
apache-kafkakafka-consumer-apiapache-kafka-connect

1 Answers

0
votes

From Kafka Connect user guide on Confluent page:

...

Kafka Connect isolates each plugin from one another so that libraries in one plugin are not affected by the libraries in any other plugins. This is very important when mixing and matching connectors from multiple providers.

A Kafka Connect plugin is:

  1. an uber JAR containing all of the classfiles for the plugin and its third-party dependencies in a single JAR file; or
  2. a directory on the file system that contains the JAR files for the plugin and its third-party dependencies.

In your case you have to put plugin jars in one folder, ex /data/pluginName not directly in /data/

More details can be found here: Installing Plugins