I am using https://hub.docker.com/r/1ambda/kafka-connect/ to install docker kafka connect on ubuntu .I am able to run it but ,I am unable to install any more connectors in it.
What I have tried
1-I tried copying the connector files from my machine to the docker containers connector folder and restarting but the api ,http://localhost:8080/connectors give empty array.
2-Also added plugin path in connect-distributed.properties . How to do this any idea ??
That image does not appear to be updated to work with plugin.path since it is using Kafka 0.10
You could use confluent-hub to install connectors like I have documented in my Connect container
https://github.com/OneCricketeer/apache-kafka-connect-docker
1-I tried copying the connector files from my machine to the docker containers connector folder and restarting but the api ,http://localhost:8080/connectors give empty array.
localhost:8083/connectors , will give you the list of active connectors. If you need to check the list of available plugins you should hit localhost:8083/connector-plugins
curl localhost:8083/connector-plugins
[{"class":"io.confluent.connect.activemq.ActiveMQSourceConnector","type":"source","version":"5.5.1"},{"class":"io.confluent.connect.elasticsearch.ElasticsearchSinkConnector","type":"sink","version":"5.5.1"},{"class":"io.confluent.connect.ibm.mq.IbmMQSourceConnector","type":"source","version":"5.5.1"},{"class":"io.confluent.connect.jdbc.JdbcSinkConnector","type":"sink","version":"5.5.1"},
{"class":"io.confluent.connect.jdbc.JdbcSourceConnector","type":"source","version":"5.5.1"},{"class":"io.confluent.connect.jms.JmsSourceConnector","type":"source","version":"5.5.1"},{"class":"io.confluent.connect.s3.S3SinkConnector","type":"sink","version":"5.5.1"},{"class":"io.confluent.connect.storage.tools.SchemaSourceConnector","type":"source","version":"5.5.1-ccs"},{"class":"io.confluent.kafka.connect.datagen.DatagenConnector","type":"source","version":"null"},{"class":"org.apache.kafka.connect.file.FileStreamSinkConnector","type":"sink","version":"5.5.1-ccs"},
{"class":"org.apache.kafka.connect.file.FileStreamSourceConnector","type":"source","version":"5.5.1-ccs"},{"class":"org.apache.kafka.connect.mirror.MirrorCheckpointConnector","type":"source","version":"1"},{"class":"org.apache.kafka.connect.mirror.MirrorHeartbeatConnector","type":"source","version":"1"},{"class":"org.apache.kafka.connect.mirror.MirrorSourceConnector","type":"source","version":"1"}]
Also, a new connector needs to be added to the mentioned plugin-path of all your worker nodes and a restart is required for all
To further troubleshoot you can try increasing the loglevel to DEBUG and check the startup logs if the connector path is being traversed You will see a log with the following format :
DEBUG Loading plugin urls: ......
Hope this helps resolve your issue :)
