Make ingresses/services publicly DNS resolvable on GKE

Question

We have a GKE cluster set up on Google Cloud and we want new ingress routes to be automatically resolvable under the according subdomain.

We have a subdomain (lets say app-dev.company.com) which already resolves to the public IP of our ingress controller (Traefik) and when I create a service "hello-world" on k8s, it should be automatically publicly resolvable under hello-world.app-dev.company.com as well.

I used to work with a cluster on Azure before, where it was simply a matter of configuring CoreDNS with stub domain and upstream DNS and it just worked after that. Now Google still uses kube-dns for some reason; I tried configuring the stub domain "app-dev.company.com" in kube-dns and installing CoreDNS (and scaling kube-dns down to 0) but that did not work and broke some other things, so I rolled it back.

I also looked into Googles Cloud DNS service (since Google recommends it), but I don't want that hard vendor lock in tbh, also I'm not sure if that even does what I want.

Can someone point me in the right direction here?

I guess I'm still not clear exactly where the issue is. Why exactly do you need a stub domain? — Gari Singh
I want the cluster to be responsible for it's own subdomains. This is the way we had it configured in Azure with no issue at all using CoreDNS. Once configured, it makes my life as a cluster user a lot easier. — Michael Niemand
Sorry for the back and forth, but trying to figure out the best solution here. I'm pretty sure you guys must be using Azure Hybrid DNS for this. So let's say you deploy an ingress route for "newapp.app-dev.company.com". How do you create a DNS record for "newapp.app-dev.company.com" ? — Gari Singh

Gari Singh Gari Singh · Accepted Answer · 2021-10-01T11:07:03

A few things:

You will need to reserve a global static IP address.
You will need to create/update your DNS record to point to that IP address.
There is a 1-1 mapping between GKE Ingress and a GCP external load balancer. This means that you cannot have multiple Ingress resources mapped to the same external IP.
If you want to have multiple applications behind the same Ingress, you can configure your Ingress resource to use path based or name based virtual hosting to route to multiple backend applications.

Reserve a static IP address:

gcloud compute addresses create ${ADDRESS} --global

Replace ${ADDRESS} with any name you like.

Annotate your Ingress resource to tell the load balancer to use the static IP created above

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: helloweb
  annotations:
    kubernetes.io/ingress.global-static-ip-name: ${ADDRESS}

Replace ${ADDRESS} with the name of the static IP created in the previous step

Configure multiple backend apps

Path-based:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: multi-app-fanout
  annotations:
    kubernetes.io/ingress.global-static-ip-name: ${ADDRESS}
spec:
  rules:
  - http:
      paths:
      - path: /*
        pathType: ImplementationSpecific
        backend:
          service:
            name: app1
            port:
              number: 8080
      - path: /app2/*
        pathType: ImplementationSpecific
        backend:
          service:
            name: app2
            port:
              number: 8080
      - path: /app3/*
        pathType: ImplementationSpecific
        backend:
          service:
            name: app3
            port:
              number: 8080

You would access your apps as follows:

app-dev.company.com
app-dev.company.com/app2
app-dev.company.com/app3

Name-based virtual hosting:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: multi-app-virtual
  annotations:
    kubernetes.io/ingress.global-static-ip-name: ${ADDRESS}
spec:
  rules:
  - host: app1.app-dev.company.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: app1
            port:
              number: 8080
  - host: app2.app-dev.company.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: app2
            port:
              number: 8080
  - host: ap32.app-dev.company.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: app3
            port:
              number: 8080

You would access your apps as follows:

app1.app-dev.company.com
app2.app-dev.company.com
app3.app-dev.company.com

Configure DNS records If you choose to do path-based routing, then you just need to add a single DNS A record which maps app-dev.company.com to the global IP address created above.

If you choose name-based virtual hosting, you'll need to create a wildcard record mapping *.app-dev.company.com to the global IP address.

See https://cloud.google.com/kubernetes-engine/docs/tutorials/configuring-domain-name-static-ip for more info.

Make ingresses/services publicly DNS resolvable on GKE

1 Answers