Cross-region load balancing + routing on Google Container Engine

4
votes

How do I achieve cross-region load balancing on Google Container Engine?

I will have one Kubernetes cluster per region in several regions and I need to route traffic from a single domain name to the geographically closest cluster.

Some options I've investigated:

  • Kubernetes LoadBalancers seem to be restricted to one cluster.
  • I'm not sure how you get Kubernetes Ingress to talk to different clusters. (It sounds like this object is backed by Compute Engine HTTP load balancers though.)
  • Compute Engine HTTP Load Balancers talking to exposed clusters sounds right, but the link I referenced seems to have some old terms like gcloud beta.
  • Instead of all this, can I actually get a Kubernetes cluster to span different regions?

Now if I want to route different URL paths to different containers within a pod, where do I do that? If it's at the Ingress or HTTP Load Balancer level, then I don't have enough granularity to address particular containers. Does that mean I would have to use a different pod + service for each different URL path?

2
dockerload-balancinggoogle-compute-enginekubernetesgoogle-kubernetes-engine

2 Answers

7
votes

Google's Network load balancing (L3) load balancing is specifically per-region (these are the load balancers that are automatically configured if you create a service of type LoadBalancer). As Alex mentioned in his answer, if you use network load balancing you will need to configure one load balancer per region and then use DNS to spread user requests to each of your load balancers.

Google's HTTP(S) load balancing is cross-region (e.g. global). This means that you get a single IP that will balance across all of your HTTP(S) backends, which can be spread across multiple clusters in multiple regions. For cross cluster load balancing, you must configure the HTTP(S) load balancer yourself as described in Is it possible to use 1 Kubernetes ingress object to route traffic to k8s services in different clusters?.

In either case, you will need to create a different service for for each URL path that you want to route to a unique backend. The services don't have to use different pods, although you may want to if they receive different amounts of traffic and you want to scale them independently.

If you use the HTTP(S) load balancer, you can define these services and the URL mapping as part of the load balancer configuration and let the HTTP(S) balancer do the request inspection / routing for you. If you use the network load balancer, then you will need to run an HTTP(S) server yourself that terminates the connection, inspects the request, and routes it to the appropriate service.

Instead of all this, can I actually get a Kubernetes cluster to span different regions?

Not out of the box. You can configure a multi-zone cluster (within a region), but we don't offer explicit support for configuring a cluster than spans regions. While you could do this manually yourself, we don't recommend it as there are many parameters baked into the cluster management software that have been tuned with the assumption of low-latency communication between the master and nodes within the cluster.

1
votes

Right now the most straightforward option is to run fungible replicas in each cluster, configure a load balancer for each cluster, and manually set up your DNS record to point to the load balancers.

That's admittedly not great, though. A better solution is being worked on for the upcoming 1.3 release: https://github.com/kubernetes/kubernetes/issues/23848