How to pull a subnet id based on the cidr block in terraform?

Question

I have create vpc and 3 subnets in a region.I have 3 services named jsc,valid,test and i have to create 3 instances for each service and i have to pass subnet id from each zone cidr block

variable "region" {
  type    = string
  default = "ap-south-1"
}
variable "ecom-cidr" {
  type    = string
  default = "10.0.0.0/16"
}

variable "azs" {
  type    = list(any)
  default = ["ap-south-1a", "ap-south-1b", "ap-south-1c"]
}

variable "private-subnets" {
  type    = list(any)
  default = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
}

variable "public-subnets" {
  type    = list(any)
  default = ["10.0.4.0/28", "10.0.5.0/28", "10.0.6.0/28"]
}


variable "instance_count" {
  type    = string
  default = 3
}

variable "service-names" {
  type    = list(any)
  default = ["valid", "jsc", "test"]

}
resource "aws_subnet" "ecom-private" {
  count                   = length(var.private-subnets)
  vpc_id                  = aws_vpc.ecom-vpc.id
  cidr_block              = element(var.private-subnets, count.index)
  availability_zone       = element(var.azs, count.index)
  map_public_ip_on_launch = false
  tags = {
    Name = "ecom-INT-${element(split("-", element(var.azs, count.index)), 2)}"
  }
}

data "aws_subnet" "priv-subnet-details" {
  vpc_id = aws_vpc.ecom-vpc.id
  count                   = length(var.private-subnets)
  filter {
    name   = "tag:Name"
    values = ["ecom-INT-${element(split("-", element(var.azs, count.index)), 2)}"] # insert values here
  }
  
}

Now i have to pass the subnet id to below aws instance resource by either lookup a Name Tag or cidr block

 resource "aws_instance" "ecom-instances" {
    
      count = 3*var.instance_count
    
      ami           = data.aws_ami.ecom.id
      instance_type = "t3.micro"
      subnet_id = <how to fetch>
      tags = {
        Name = "ecom-${element(var.service-names, count.index)}-service"
        Service = "${element(var.service-names, count.index)}"
      }
vpc_security_group_ids = [aws_security_group.ecom-sg["${element(var.service-names, count.index)}"].id]

Subnet details from data source is as below

subnet-details = [
      + {
          + arn                             = "arn:aws:ec2:ap-south-1:114712064551:subnet/subnet-07fede319193c390d"
          + assign_ipv6_address_on_creation = false
          + availability_zone               = "ap-south-1a"
          + availability_zone_id            = "aps1-az1"
          + available_ip_address_count      = 251
          + cidr_block                      = "10.0.1.0/24"
          + customer_owned_ipv4_pool        = ""
          + default_for_az                  = false
          + filter                          = [
              + {
                  + name   = "tag:Name"
                  + values = [
                      + "ecom-INT-1a",
                    ]
                },
            ]
          + id                              = "subnet-07fede319193c390d"
          + ipv6_cidr_block                 = null
          + ipv6_cidr_block_association_id  = null
          + map_customer_owned_ip_on_launch = false
          + map_public_ip_on_launch         = false
          + outpost_arn                     = ""
          + owner_id                        = "114712064551"
          + state                           = "available"
          + tags                            = {
              + "Name" = "ecom-INT-1a"
            }
          + vpc_id                          = "vpc-0fd6cb4be13de15e8"
        },
      + {
          + arn                             = "arn:aws:ec2:ap-south-1:114712064551:subnet/subnet-0680314588bcd64e6"
          + assign_ipv6_address_on_creation = false
          + availability_zone               = "ap-south-1b"
          + availability_zone_id            = "aps1-az3"
          + available_ip_address_count      = 251
          + cidr_block                      = "10.0.2.0/24"
          + customer_owned_ipv4_pool        = ""
          + default_for_az                  = false
          + filter                          = [
              + {
                  + name   = "tag:Name"
                  + values = [
                      + "ecom-INT-1b",
                    ]
                },
            ]
          + id                              = "subnet-0680314588bcd64e6"
          + ipv6_cidr_block                 = null
          + ipv6_cidr_block_association_id  = null
          + map_customer_owned_ip_on_launch = false
          + map_public_ip_on_launch         = false
          + outpost_arn                     = ""
          + owner_id                        = "114712064551"
          + state                           = "available"
          + tags                            = {
              + "Name" = "ecom-INT-1b"
            }
          + vpc_id                          = "vpc-0fd6cb4be13de15e8"
        },
      + {
          + arn                             = "arn:aws:ec2:ap-south-1:114712064551:subnet/subnet-04f6ab1d3612c4f48"
          + assign_ipv6_address_on_creation = false
          + availability_zone               = "ap-south-1c"
          + availability_zone_id            = "aps1-az2"
          + available_ip_address_count      = 251
          + cidr_block                      = "10.0.3.0/24"
          + customer_owned_ipv4_pool        = ""
          + default_for_az                  = false
          + filter                          = [
              + {
                  + name   = "tag:Name"
                  + values = [
                      + "ecom-INT-1c",
                    ]
                },
            ]
          + id                              = "subnet-04f6ab1d3612c4f48"
          + ipv6_cidr_block                 = null
          + ipv6_cidr_block_association_id  = null
          + map_customer_owned_ip_on_launch = false
          + map_public_ip_on_launch         = false
          + outpost_arn                     = ""
          + owner_id                        = "114712064551"
          + state                           = "available"
          + tags                            = {
              + "Name" = "ecom-INT-1c"
            }
          + vpc_id                          = "vpc-0fd6cb4be13de15e8"
        },
    ]

Could you please guide me

Tried the below method but since for_each is containing the subnet id's which compute after apply,terraform is throwing error.So i want to directly pass the subnet-id instead of using for each

locals {
  instance_configs = tomap({
    for idx,pair in setproduct(var.service-names, data.aws_subnet.priv-subnet-details.*.id) :
    "${pair[0]}-${pair[1]}" => {
      service_name = pair[0]
      subnet = pair[1]
    }
  })
}

resource "aws_instance" "ecom-instances" {
   for_each = local.instance_configs
  ami           = data.aws_ami.ecom.id
  instance_type = "t3.micro"
  tags = {
    Name = "ecom-${each.value.service_name}-service"
    Service = each.value.service_name
  }
vpc_security_group_ids = [aws_security_group.ecom-sg[each.value.service_name].id]
subnet_id              = each.value.subnet
}

Answer 1

From your description it sounds like your problem could be modeled more directly as a map of objects where each element represents the settings for a particular availability zone:

variable "availability_zones" {
  type = map(object({
    private_subnet = string
    public_subnet  = string
  }))
}

You could then assign this a value like the following in order to clearly indicate which CIDR ranges belong to which availability zones:

  availability_zones = {
    ap-south-1a = {
      private_subnet = "10.0.1.0/24"
      public_subnet  = "10.0.4.0/28"
    }
    ap-south-1b = {
      private_subnet = "10.0.2.0/24"
      public_subnet  = "10.0.5.0/28"
    }
    ap-south-1c = {
      private_subnet = "10.0.3.0/24"
      public_subnet  = "10.0.6.0/28"
    }
  }

Now you have a data structure that directly matches the expectations of for_each to create these subnets. For example, for your private subnets:

resource "aws_subnet" "ecom_private" {
  for_each = var.availability_zones

  vpc_id                  = aws_vpc.ecom-vpc.id
  cidr_block              = each.value.private_subnet
  availability_zone       = each.key
  map_public_ip_on_launch = false
  tags = {
    Name = "ecom-INT-${split("-", each.key)[2]}"
  }
}

An advantage of this approach is that aws_subnet.ecom_private is now a map from availability zones to subnet objects. As I mentioned in my answer to your previous question, a particular Terraform configuration should typically not read back with a data block the same objects it's also managing with resource blocks; instead, we refer to the results of the resource blocks directly so that Terraform can see how the objects are connected and thus produce the correct order of operations.

For the second part of your problem it sounds like you want to declare an aws_instance instance per service and per subnet. To achieve that with for_each we need to derive a new data structure that has one element per combination of service and subnet. To find all combinations of two sets I'd typically use the setproduct function, which in this case we might use like this:

locals {
  service_subnets = {
    for pair in setproduct(var.service_names, values(aws_subnet.ecom_private)) :
    "${pair[0]}:${pair[1].availability_zone}" => {
      service_name = pair[0]
      subnet       = pair[1]
    }
  }
}

This expression creates a mapping from compound keys containing both a service name and an availability zone name to objects containing both the service name and one of the subnet objects. This data structure is now of a suitable shape to declare aws_instance per element of this collection. Note that this is different than what you tried because the keys are built only from data defined directly in the configuration: service names and availability zone names. Your example didn't work because you tried to use the subnet id, which EC2 doesn't decide until after the subnet has already been created.

resource "aws_instance" "ecom" {
  for_each = local.service_subnets

  ami           = data.aws_ami.ecom.id
  instance_type = "t3.micro"
  subnet_id     = each.value.subnet.id

  vpc_security_group_ids = [aws_security_group.ecom_sg[each.value.service_name].id]

  tags = {
    Name    = "ecom-${each.value.service_name}-service"
    Service = each.value.service_name
  }
}

These instances will therefore have addresses like this, where the keys include only data that you've defined directly in your configuration:

• aws_instance.ecom["valid:ap-south-1a"]
• aws_instance.ecom["valid:ap-south-1b"]
• aws_instance.ecom["valid:ap-south-1c"]
• aws_instance.ecom["jsc:ap-south-1a"]
• aws_instance.ecom["jsc:ap-south-1b"]
• aws_instance.ecom["jsc:ap-south-1c"]
• aws_instance.ecom["test:ap-south-1a"]
• aws_instance.ecom["test:ap-south-1b"]
• aws_instance.ecom["test:ap-south-1c"]