How to configure MongoDB official source connector for Kafka Connect running on a kubernetes cluster

1
votes

My Kafka cluster runs on kubernetes and I am using a custom image to run Kafka Connect with required mongoDB official source and sink connectors.

My mongoDB instance also runs on kubernetes. My issue is, I am unable to connect my live DB with Kafka Connect.

My connector config currently looks like this,

curl -X PUT \
     -H "Content-Type: application/json" \
     --data '{
               "connector.class":"com.mongodb.kafka.connect.MongoSourceConnector",
                    "tasks.max": "1",
                    "connection.uri": "mongodb://192.168.190.132:27017,192.168.190.137:27017",
                    "database": "tractor",
                    "collection": "job",
                    "topic.prefix": "testing-mongo"
                    
          }' \
     http://10.108.202.171:8083/connectors/mongo_source_job/config
2
mongodbkubernetesapache-kafka-connect
Can you edit your post with a snippet from the kafka connector logs on it's attempt to connect to mongo? - alex

2 Answers

2
votes

Thanks for your reply. The issue was the stemming from TLS. I modified my config as follows,

"connection.uri": "mongodb://192.168.190.132:27017,192.168.190.137:27017/?tlsInsecure=true"

Its working now!

1
votes

Can you try connecting to MongoDB service using the service name?

kubectl get service -n <namespace>

Use the above to get the services in the namespace of MongoDB and use the service name instead of the Ip's you have and see if that works?