Kubernetes host ingress with nested subdomains (GCP)

Question

I am trying to setup a K8s deployment where ingress's controllers can define a service as a subdomain. i.e. app1 can define itself to get traffic from app1.sub.domain.io in its ingress config.

I have a DNS A Record *.sub.domain.io that points to a Load Balancer. That load balancer is pointing to the cluster's instance group.

So if I am right all traffic that goes to anything at sub.domain.io will land inside the cluster and just need to route said traffic.

Below are the k8 configs, which has a pod, a service and an ingress. The pods are healthy and working, I believe the service isn't required but will want other pods to talk to it via internal DNS so it's added.

The ingress rules have a host app1.sub.domain.io, so in theory, curl'ing app1.sub.domain.io should follow: DNS -> Load Balancer -> Cluster -> Ingress Controller -> Pod

At the moment when I try to hit app1.sub.domain.io it just hangs. I have tried not having service, making external-name service and doesn't work.

I don't want to go down the route of using the loadBalancer ingress as that makes a new external IP that needs to be applied to DNS records manually, or with a nasty bash script that waits for services external IP and runs GCP command, and we don't want to do this for each service.

Ref links:https://kubernetes.io/docs/concepts/services-networking/ingress/#name-based-virtual-hosting

Deployment

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: app1
  namespace: default
  labels:
    app: app1
spec:
  replicas: 3
  selector:
    matchLabels:
      app: app1
  template:
    metadata:
      labels:
        app: app1
    spec:
      containers:
      - image: xxxx:latest
        name: app1
        ports:
        - containerPort: 80
        env:
          - name: NODE_ENV
            value: production

Service

---
kind: Service
apiVersion: v1
metadata:
  name: app1
  labels:
    app: app1
spec:
  ports:
    - port: 80
      targetPort: 80
  selector:
    app: app1
  type: ClusterIP

Ingress

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: app1-ingress
  labels:
    app: app1
spec:
  rules:
  - host: app1.sub.domain.io
    http:
      paths:
      - backend:
          serviceName: app1
          servicePort: 80

Once everything is deployed if you query kubectl get pods,services,ingresses -l app=app1

NAME                       READY     STATUS    RESTARTS   AGE
po/app1-6d4b9d8c5-4gcz5    1/1       Running   0          20m
po/app1-6d4b9d8c5-m4kwq    1/1       Running   0          20m
po/app1-6d4b9d8c5-rpm9l    1/1       Running   0          20m

NAME        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
svc/app1    ClusterIP   x.x.x.x         <none>        80/TCP    20m

NAME                HOSTS                   ADDRESS   PORTS     AGE
ing/app1-ingress    app1.sub.domain.io                80        20m

----------------------------------- Update -----------------------------------

Currently doing this, not ideal. Have global static IP that's assigned to a DNS record.

---
kind: Service
apiVersion: v1
metadata:
  name: app1
  labels:
    app: app1
spec:
  type: NodePort
  selector:
    app: app1
  ports:
    - port: 80
      targetPort: 80

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: app1-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: app1-static-ip
  labels:
    app: app1-static-ip
spec:
  backend:
    serviceName: app1
    servicePort: 80

I think that the workaround in your edit is the solution. What would you like to improve? — Matt-y-er
Did you get to resolve this? We wanted to setup a staging domain at app.stg.xxx.com. We use app-stg.xxx.com as a workaround until we have more info on this. — borislemke

Christina A Christina A · Accepted Answer · 2018-03-13T16:46:27

Try adding path to your Ingress:

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: app1-ingress
  labels:
    app: app1
spec:
  rules:
  - host: app1.sub.domain.io
    http:
      paths:
      - backend:
          serviceName: app1
          servicePort: 80
        path: /

If that doesn't work, please post the output of describe service and describe ingress.

Do you have an Ingress Controller? Traffic should go LB-> Ingress Controller-> Ingress-> Service ClusterIP-> Pods

Kubernetes host ingress with nested subdomains (GCP)

2 Answers