Extract an unassigned subnet CIDR block from a fragmented VPC

Question

Lets suppose you have a VPC with a given cidr block. Let's say 10.0.0.0/16. Now let's say you have assigned about 20 subnets from the VPC. And those subnets are neither contiguous nor of the same width. ie one given subnet is 10.0.0.7/27, and some other subnet is 10.0.128.0/25 and so forth.

What if I want to carve a subnet with 32 IP addresses (or n ip addresses). How do I get its cidr block?

Are there any good aws libraries, terraform libraries or anyone has encountered this problem and solved it. I want to be able to create new subnets with given widths.

I am aware of ec2.describe-subnets, jq, ipcalc and so forth.

Answer 1

I'd use the Python netaddr package to solve your problem:

from netaddr import *
import math

cidr = '10.0.0.0/16' # Your VPC's CIDR block

assigned = [ # Networks you've already used
  '10.0.0.7/27',
  '10.0.128.0/25'
]

needed_ips = 32 # Number of IP addresses needed

available = IPSet([cidr]) - IPSet(assigned)
needed_prefix = 32 - math.ceil(math.log2(needed_ips))
for net in available.iter_cidrs():
  if net.prefixlen <= needed_prefix:
    print(next(net.subnet(needed_prefix, 1)))
    break

Answer 2

If using Python, the boto3 package along with the built-in ipaddress module together are sufficient to list unused candidate subnets of a given prefix length for a specified VPC.

This script was tested with Python 3.9, but it should work with Python ≥3.6. Define the values of VPC_NAME and REQUIRED_PREFIX_LEN. It will print all relevant unused subnets of the required length, although you can choose to adapt it to print fewer. From the printed candidates, you can carefully pick the ones which minimize further fragmentation.

import ipaddress
from typing import List

import boto3

# Customize these parameters:
VPC_NAME = 'my-vpc'
REQUIRED_PREFIX_LEN = 23

# Get VPC CIDR
vpcs = boto3.client('ec2').describe_vpcs(Filters=[{'Name': 'tag:Name', 'Values': [config.EC2_VPC_NAME]}])['Vpcs']
assert len(vpcs) == 1
vpc = vpcs[0]
vpc_cidr = vpc['CidrBlock']
vpc_net = ipaddress.ip_network(vpc_cidr)
assert vpc_net.prefixlen < REQUIRED_PREFIX_LEN
print(f'VPC {VPC_NAME} has CIDR {vpc_cidr}.')


def print_subnets(networks: List, description: str) -> None:
    print(f"\nThe {len(networks)} {description} subnets are: {' '.join(map(str, networks))}")


# Get used subnets
used_subnets = boto3.client('ec2').get_paginator('describe_subnets').paginate(Filters=[{'Name': 'vpc-id', 'Values': [vpc['VpcId']]}]).build_full_result()['Subnets']
used_subnets = [ipaddress.ip_network(s['CidrBlock']) for s in used_subnets]
print_subnets(used_subnets, 'used')
collapsed_used_subnets = list(ipaddress.collapse_addresses(used_subnets))
print_subnets(collapsed_used_subnets, 'collapsed used')

# Get unused subnets
unused_subnets = list(vpc_net.subnets(new_prefix=REQUIRED_PREFIX_LEN))
for used_subnet in collapsed_used_subnets:
    unused_subnets = [unused_subnet for unused_subnet in unused_subnets if not unused_subnet.overlaps(used_subnet)]
print_subnets(unused_subnets, 'relevant unused')
collapsed_unused_subnets = list(ipaddress.collapse_addresses(unused_subnets))
print_subnets(collapsed_unused_subnets, 'relevant collapsed unused')

It is not an algorithmically efficient script, but unless you are doing this at scale, it doesn't have to be.