resolve service hostname in ECS fargate task

Question

I am trying to automate my ECS fargate cluster making using terraform.

I have a SpringBoot project with microservices containerized, and I am putting these images in a single task definition for an ECS service for the backend.

The ECS cluster is initially running, but Kafka is getting stopped with the error :

ERROR Unable to resolve address: zookeeper:2181
(org.apache.zookeeper.client.StaticHostProvider)

I have given the image for zookeeper(bitnami) and used the KAFKA_CFG_ZOOKEEPER_CONNECT environment variable too.

EDIT : My task definition:

resource "aws_ecs_task_definition" "this" {
  family                   = local.application_name
  requires_compatibilities = [local.launch_type]
  execution_role_arn       = data.aws_iam_role.ecs_task_execution_role.arn
  network_mode             = "awsvpc"
  cpu                      = "4096"
  memory                   = "30720"
  container_definitions = jsonencode([
    {
      name      = "zookeeper"
      image     = "docker.io/bitnami/zookeeper:latest"
      essential = true
      cpu       = 512
      memory    = 1024
      portMappings = [
        {
          containerPort = 2181
          hostPort      = 2181
        }
      ]
      "environment" : [
        { "name" : "ALLOW_ANONYMOUS_LOGIN", "value" : "yes" },
        { "name" : "ZOO_LISTEN_ALLIPS_ENABLED", "value" : "yes" }
      ]
    },
    {
      name      = "kafka-server"
      image     = "docker.io/bitnami/kafka:latest"
      essential = true
      cpu       = 512
      memory    = 1024
      portMappings = [
        {
          containerPort = 9092
          hostPort      = 9092
        }
      ]
      depends_on =  [
        "zookeeper"
      ]
      "environment" : [
        { "name" : "KAFKA_CFG_ZOOKEEPER_CONNECT", "value" : "zookeeper:2181" },
        { "name" : "ALLOW_ANONYMOUS_LOGIN", "value" : "yes" },
        { "name" : "ALLOW_PLAINTEXT_LISTENER", "value" : "yes" }
      ],
    },
    {
      name      = "email-service"
      image     = "my-email-image"
      essential = true
      cpu       = 512
      memory    = 1024
      portMappings = [
        {
          containerPort = 8090
          hostPort      = 8090
        }
      ]
      "environment" : [
        { "name" : "EMAIL_URL", "value" : "email-service" },
        { "name" : "EMAIL_PORT", "value" : "8090" },
        { "name" : "KAFKA_URL", "value" : "kafka-server" },
        { "name" : "KAFKA_PORT", "value" : "9092" },
      ]
    }
  ])
}

If you give a name like this, it's using DNS. I think you should change zookeeper to the FQDN of your zookeeper deployment. I doubt it's simply zookeeper. Can you share your task definition? — The Fool
@TheFool I have included task definition in question edit and I think I might have used depends_on incorrectly but was not getting any error in terraform plan . — Rohit Bhatt
Your issue might be that you have network mode set to awsvpc. I am not sure if you can simply use the service name like that anymore, if you do that. Can you try to set it to bridge once and see if it works? Then you know at least where to look for further configuration if it's that. docs.aws.amazon.com/AmazonECS/latest/developerguide/… — The Fool
I think this sentence is the key: "Additionally, containers that belong to the same task can communicate over the localhost interface." Can you try to set KAFKA_CFG_ZOOKEEPER_CONNECT to localhost:2181 ? docs.aws.amazon.com/AmazonECS/latest/userguide/… — The Fool
What exactly are the requirements? Like I said, the data isn't persistent, so you might as well just use Spring's internal EventListener classes or some other embedded event bus library — OneCricketeer

The Fool The Fool · Accepted Answer · 2021-08-22T17:28:29

AS written in the documentation:

Additionally, containers that belong to the same task can communicate over the localhost interface.

So my suggestion is to use localhost instead of the service names. For example, you want to do it for Kafka but also for every other service, such as the email service.

{ "name" : "KAFKA_CFG_ZOOKEEPER_CONNECT", "value" : "localhost:2181" },

resolve service hostname in ECS fargate task

1 Answers