Terraform Finding IP address available across subnets

Question

We are provisioning a fairly large set of servers at one go (57 per environment per deployment). The IP addresses we are getting per subnet are 32 so, a total of 96 IPs per AWS account, and I want to utilize it efficiently.

we are using IP rotation across subnets using this element(split(",",local.ec2_subnet),count.index)
Say because or random termination of instances. We left with 57 IPs out of 96 but not evenly distributed across subnets. Subnet-1a :7 Subnet-1b :25 Subnet-1c :25
now, when I try to provision the next set. The terraform failed with an error for the non-availability of the IP address in the subnet. It was assigned on a rotational basis. (without feedback from AWS)
Is there a better way to know the subnet with an available IP address and assign it accordingly?

@ydaetskcoR Adding code for re-production: 3 Types of Servers : Just create below mentioned 1+8+48=57 EC2s.

resource "aws_instance" "gs_instance" {
    count    =  1
    subnet_id   = element(split(",",local.ec2_subnet),2)
    . . .
 }
resource "aws_instance" "Head_instance" {
    count    =  8
    subnet_id   = element(split(",",local.ec2_subnet),2)
    . . .
 }
resource "aws_instance" "compute_instance" {
    count    =  48
    subnet_id   = element(split(",",local.ec2_subnet),2)
    . . .
 }

say the distribution goes as below mentioned table. and terminate only few of them based on the subnet they are in. so that you get the number of available IP addresses

                     Subnet-1a(32)     Subnet-1b(32)       Subnet-1c(32) 
provision           32-19(used)=13    32-19(used)=13      32-19(used)=13    
Termination         6=(available 7)  12=(available 25)   12=(available 25)

now with above Ip addresses availability if I provision next 57 nodes, it's going to fail for subnet-1a after 7 EC2(12 will fail because of un availability of IP address but actually across 3-subnets there are 57 Ip address available. Hope this info is enough to reproduction.

Can you please edit your question to include some Terraform code as a minimal reproducible example that shows what you're trying and how you are ending up with this disparity in spread across the AZs? — ydaetskcoR

Steven Viljoen Steven Viljoen · Accepted Answer · 2020-10-09T07:26:13

Perhaps try dynamically fetching the appropriate subnet first with a Data Source (aws_subnet_ids) using the 'available-ip-address-count' filter and then pass that subnet's id into the provisioning module.

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet_ids https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeSubnets.html

Terraform Finding IP address available across subnets

1 Answers