Errors while setting up a Kafka connector pipeline
Enviroment overview:
Docker containers
- docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=mypw -d mysql:latest
- docker run -d --name Elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" Elasticsearch: 7.9.2
Operating System
- WSL2 ON - Windows-10 Version 1909 (OS build 18363.1139)
Kafka Version
- confluent-6.0.0
Hello I’m reading Kafka: The Definitive Guide by Neha Narkhede, Gwen Shapira, and Todd Palino I’ve reached the Connector Example: MySQL to Elasticsearch chapter (p.146) and i'm following the instructions to create a pipeline from a MySql source to a Elasticsearch sink. I have made some diversions from the instructions
- I created the mysql and Elasticsearch connectors using mvn package instead of mvn build
- I placed the said connectors in a folder named C:\Users\ROY\confluent-6.0.0\share\kafka tougher with some other connectors I’ve downloaded. I set the plugin.path vaiable in connect-distributed.properties to: plugin.path=C://Users//ROY//confluent-6.0.0//share//kafka,/mnt/c/Users/ROY/confluent-6.0.0/share/kafka
- I'm using mysql and Elasticsearch as docker containers.
The mysql connector works fine and reads data into a topic, but when I try to create the Elasticsearch connector I get the following error:
...
(io.confluent.connect._Elasticsearch_._Elasticsearch_SinkConnectorConfig:354)
[2020-10-16 12:22:27,170] ERROR WorkerSinkTask{id=elastic-login-connector-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:187)
java.lang.NoClassDefFoundError: io/searchbox/action/Action
at io.confluent.connect._Elasticsearch_._Elasticsearch_SinkTask.start(_Elasticsearch_SinkTask.java:74)
at io.confluent.connect._Elasticsearch_._Elasticsearch_SinkTask.start(_Elasticsearch_SinkTask.java:48)
at org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart(WorkerSinkTask.java:302)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:193)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:185)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:235)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.ClassNotFoundException: io.searchbox.action.Action
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:104)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 11 more
...
Can someone assist resolving the issue and give and explanation to the error?
Thank you
Roy
