For a GKE Kubernetes Cluster w/Alias what will a custom Secondary Range Name default to?

Question

I am running multiple GKE Clusters on Google Cloud behind a multi-cluster ingress the setup of which took some tinkering.

That being said by the time I got it sorted out I had used up my 5 subnets specifically on the us-east1 region. I don't need more than 5 but... apparently deleting a Cluster does NOT also delete the Subnet that was created with it.

Therefore — I need to manually delete the unused subnets, and to do that I need to know the subnet names.

My issue is that I can't find any where in GCP's docs that describes what the name of a subnet will be after being created with a new GKE Cluster (where the subnet name is not specified).

What would the Subnet Name be with the following Create Cluster command (for example):

gcloud container clusters create example-cluster --num-nodes=1 --region us-east1-c \
  --machine-type g1-small \
  --enable-ip-alias --cluster-ipv4-cidr=10.2.0.0/21 \
  --services-ipv4-cidr=10.6.0.0/19

The following gcloud Subnet docs (https://cloud.google.com/vpc/docs/configure-alias-ip-ranges) described how to list the subnets — but on running: gcloud compute networks list I get only the default response:

NAME     SUBNET_MODE  BGP_ROUTING_MODE  IPV4_RANGE  
default  AUTO         REGIONAL

I know that there are multiple subnets created within this region because attempting to create another one (with new cluster) gives me the following error:

'Retry budget exhausted (5 attempts): Google Compute Engine: Exceeded maximum supported number of secondary ranges per subnetwork: 5.' endTime: u'2019-01-10T22:16:14.496871616Z'

I am open to additional solutions that allow me to properly list all subnets in a region BUT my question is how does GKE name a Subnet that is created without a name?

Necevil Necevil · Accepted Answer · 2019-01-10T22:55:27

OK. No sooner do I give up and post the question than I find out there is a gcloud beta feature that can list container network subnets:

gcloud beta container subnets list-usable

You can find the full docs over here: https://cloud.google.com/sdk/gcloud/reference/beta/container/subnets/list-usable

Basically with the stipulated Cluster Create command in the Question GKE will create two secondary ranges, one for Pods and one for Services that look like:

┌───────────────────────────────────────┬───────────────┬──────────┐
│          SECONDARY_RANGE_NAME         │ IP_CIDR_RANGE │  STATUS  │
├───────────────────────────────────────┼───────────────┼──────────┤
│ gke-example-cluster-pods-241ef819     │ 10.0.0.0/21   │ unusable │
│ gke-example-cluster-services-241ef819 │ 10.4.0.0/19   │ unusable │

I am not exactly sure what the characters tagged on the end of the secondary range name are but I assume they are uniquely generated for each Cluster as they are all different across my multi region clusters.

One odd thing that I found out based on the above command is that you can only have two Clusters that are routable using VPC Native networking in each region.

While this does work for my use case (and I agree it should work for most) I just thought it was interesting. As an aside using the above command I see that the ranges created with GKE ARE deleted on cluster deletion BUT you can only have a maximum 5 subnet ranges per each network (in this case default).

My guess is you could create an additional subnetwork and then have additional secondary ranges within that — but I haven't tried this as of now.

For a GKE Kubernetes Cluster w/Alias what will a custom Secondary Range Name default to?

1 Answers