Will VMs Fill Secondary IP Ranges?

Question

I am trying to design very large scale networks in GCP, specifically porting this network framework to GCP from AWS.

In AWS, subnets are tied to a specific availability zone. In GCP, subnets span all availability zones.

locals {
  # for network 0, layer index 0, occupy 10.0.0.0-10.0.9.255
  primary_ip_range = "10.${var.network_id}.${var.layer_index * 10}.0/23"
  secondary_ip_ranges = [
    { range_name = "${var.name}-0", ip_cidr_range = "10.${var.network_id}.${(var.layer_index * 10) + 2}.0/23" },
    { range_name = "${var.name}-1", ip_cidr_range = "10.${var.network_id}.${(var.layer_index * 10) + 4}.0/23" },
    { range_name = "${var.name}-2", ip_cidr_range = "10.${var.network_id}.${(var.layer_index * 10) + 6}.0/23" },
    { range_name = "${var.name}-3", ip_cidr_range = "10.${var.network_id}.${(var.layer_index * 10) + 8}.0/23" },
  ]
}

resource "google_compute_subnetwork" "default" {
  region = "${var.region}"

  name = "${var.name}"
  description = "The ${title(var.name)} layer subnet in the ${var.zone} Titan network."
  network = "${var.vpc_id}"
  private_ip_google_access = true

  ip_cidr_range = "${local.primary_ip_range}"
  secondary_ip_range = ["${local.secondary_ip_ranges}"]
}

The Google documentation seems to call these secondary CIDR blocks as "alias ranges" to be assigned to containers running on a VM. However, I'm more interested in how these secondary CIDR blocks will interact with normal VMs.

If I exhaust the primary CIDR block (10.0.0.0/23), will hosts overflow into another CIDR block? Or, is the primary range the only range used by VMs in the network?

NOTE: In the AWS implementation, Titan allows between 1 and 5 adjacent /23s as described above, limited by the number of AWS availability zones in the region.

This gives between 512 and 2560 available hosts in a "layer," which is a logical grouping of subnets for a similar purpose. In AWS, it's easy to work with multiple subnets, splaying instances across them or having an autoscaling group or load balancer place nodes evenly across all subnets.

In GCP, you can only specify one zone for a managed instance group, so the idea seems to be that you should have one managed instance group per zone within a subnet.

Confused yet? I'm trying to translate concepts from AWS to GCP and the underlying pieces are different so I'm trying to map things as closely as reasonable.

Muhammad Ebaduddin Muhammad Ebaduddin · Accepted Answer · 2018-07-19T18:34:17

Unfortunately, the VM still needs an IP address in the primary range. If you run out of addresses, in that range you can resize the subnet following the directions from the document below [1]

[1] https://cloud.google.com/sdk/gcloud/reference/compute/networks/subnets/expand-ip-range

Will VMs Fill Secondary IP Ranges?

1 Answers