How do I map external traffic to the internal service mesh on GCP Traffic Director?

Question

I've set up a simple GKE cluster hooked to GCP Traffic Director with the Traffic Director setup with automatic Envoy injection tutorial.

The next step is how do I map external traffic into the Traffic Director backend service, which is only internal?

Basically, my goal is to have an external load balancer with an IP address that takes outside traffic and routes it to the Traffic Director service mesh to split traffic between different Network Endpoint Groups.

I tried the following:

Create an external load balancer manually in Network Services -> Load Balancing --> However the list of Backends does not include the Traffic Director backend service so I can't create one to have an external IP and redirect it to the internal service mesh.

Install the NGINX ingress controller chart and install an ingress controller via .yaml that maps to the k8s cluster service --> This creates an external load balancer but it simply goes directly to the service instead of through Traffic Director

Ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-nginx-ingress
  annotations:
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "60"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "60"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
    nginx.ingress.kubernetes.io/send-timeout: "60"
    nginx.ingress.kubernetes.io/proxy-body-size: 1M
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  rules:
    - host: my-host-name.hostname.com
      http:
        paths:
          - path: "/"
            backend:
              serviceName: service-test
              servicePort: 80

Service:

apiVersion: v1
kind: Service
metadata:
  name: service-test
  annotations:
    cloud.google.com/neg: '{"exposed_ports":{"80":{"name": "service-test-neg"}}}'
spec:
  ports:
  - port: 80
    name: service-test
    protocol: TCP
    targetPort: 8000
  selector:
    run: app1
  type: ClusterIP

Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: app1
  name: app1
spec:
  replicas: 1
  selector:
    matchLabels:
      run: app1
  template:
    metadata:
      labels:
        run: app1
    spec:
      containers:
      - image: gcr.io/kubernetes-e2e-test-images/serve-hostname-amd64:1.1
        name: app1
        command:
        - /bin/sh
        - -c
        - /serve_hostname -http=true -udp=false -port=8000
        ports:
        - protocol: TCP
          containerPort: 8000

The deployment and service above is taken directly from the tutorial.

There seems to be a concept in the official documentation for Handling ingress traffic using a second-level gateway at the edge of your mesh, but it's only conceptual and does not provide how to actually do it.

How do I map external traffic using an external load balancer into a GCP Traffic Director-managed service mesh for advanced traffic configuration into GKE?

suren suren · Accepted Answer · 2021-03-05T10:48:09

Traffic Director is not an endpoint to point to for routing. It is the "control plane" of your service mesh.

So you would configure your routing rules from GCP, and Traffic Director would configure your sidecars as expected. But eventually your Load Balancer should point to an Instance Group or Network Endpoint Group, not to Traffic Director.

EDIT

Traffic Director is not the one getting configured, but the one configuring. It configures the Envoy sidecars. These are L7 proxies, so the URL mapping happens on the proxies.

The Endpoint Group will be a group of IP addresses of pods. Since the pod ranges of the cluster have been added to the subnetwork; as IP alias, the VPC is capable of pulling any IP address from this range, group it, and make a backend for a HTTP load balancer on GCP.

Basically, Traffic Director is Istio, but with control plane decoupled to GCP.

How do I map external traffic to the internal service mesh on GCP Traffic Director?

1 Answers