0
votes

We've got a managed Kafka setup (Confluent platform, Kafka connect 5.5.1), streaming data from ~40 topics across 8 to 10 connectors. A few weeks ago I noticed that for some of those topics, we don't have any consumers assigned. The consumers which should be reading from or writing to those topics are ones that our org has written and have not changed in months.

Looking through our connector hosts (AWS EC2 instances) I actually cannot see where our connector JAR files exist - which surprises me a lot. We've got all the other connectors there, and when I used confluent hub to install the BigQuery connector that got put under /usr/share/java as one would expect.

Where should home-grown connectors live on the filesystem?

For the record, when I query :8083 using the appropriate calls I can see the connector and it does have an allegedly-running task.

1

1 Answers

1
votes

They are picked from the Java CLASSPATH and plugin.path

As for where they should exist, is somewhere that the user account running the connect process has access to read those files.