kafka-connect on ECS timeouts when connecting to MSK

Question

i deployed the kafka-connect docker image(confluentinc/cp-kafka-connect-base:6.0.1) into ECS / fargate, assigned a security group to my ECS service that permits both incoming zooper keeper and kafka bootstrap server traffic (both plain text and TLS) as well as an IAM role that permits my ECS tasks to run kafka actions against the MSK cluster but still the connect cluster is timing out when its trying get the list of brokers from MSK cluster.

Both kafka connect ECS service and MSK cluster are on the same private subnets in AWS.

Security group code

      "Type": "AWS::EC2::SecurityGroup",
      "Properties": {
        "GroupDescription": "kakfa-connect-sg",
        "SecurityGroupEgress": [
          {
            "CidrIp": "0.0.0.0/0",
            "Description": "Allow all outbound traffic by default",
            "IpProtocol": "-1"
          }
        ],
        "SecurityGroupIngress": [
          {
            "CidrIp": "0.0.0.0/0",
            "Description": "Kafka Bootstrap Server Plaintext",
            "FromPort": 9092,
            "IpProtocol": "tcp",
            "ToPort": 9092
          },
          {
            "CidrIp": "0.0.0.0/0",
            "Description": "Kafka Bootstrap Server TLS",
            "FromPort": 9094,
            "IpProtocol": "tcp",
            "ToPort": 9094
          },
          {
            "CidrIp": "0.0.0.0/0",
            "Description": "ZooKeeper TLS",
            "FromPort": 2182,
            "IpProtocol": "tcp",
            "ToPort": 2182
          },
          {
            "CidrIp": "0.0.0.0/0",
            "Description": "ZooKeeper Plaintext",
            "FromPort": 2181,
            "IpProtocol": "tcp",
            "ToPort": 2181
          }
        ],
        "VpcId": "vpc-id"
      }```

IAM role code

```{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "kafka:*",
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}```

Is there anything i might be missing?

my kafka connect cluster was using PLAIN TEXT to talk to the MSK cluster, i fixed this but am still getting timeouts for some reason i can't get DEBUG logs for AdminClient class even when i set CONNECT_LOG4J_LOGGERS=org.apache.kafka.clients.admin=DEBUG — austiine

austiine austiine · Accepted Answer · 2021-01-11T22:16:13

The security group settings on the MSK cluster was not permitting traffic from my kafka connect cluster.

kafka-connect on ECS timeouts when connecting to MSK

1 Answers