gke ingress unable to route traffic to services

Question

I am testing gke ingress to route traffic to two different services. My deployment consists of a basic web container that deploys a blue web page default and a green web page. I am able to get a response back Essentially the "/" works with either blue or green deployment. But when i go to http:///green i get a 404 response. I have tested the same with "/" as green deployment and it displays a green web page. But if i go http:///blue it results in a 404 response,

I have verified my containers are working properly by attaching a load balancer directly to them. I am following this how to guide in gke to setup a similar environment. GKE Ingress How to guide

Any help on what i am missing would greatly help me understand better what is going on and why my gke load balancer is unable to route traffic.

green deployment file

apiVersion: apps/v1
kind: Deployment
metadata: 
  name: myapp-green
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myapp
      version: green
  template:
    metadata:
      name: myapp
      labels:
        app: myapp
        version: green
    spec:
      containers:
        - name: myapp
          image: gcr.io/ultra-welder-300122/myapp:green
          imagePullPolicy: Always
          resources:
            requests:
              cpu: "100m"
              memory: "100Mi"
          ports:
          - containerPort: 8080

blue Deployment file

apiVersion: apps/v1
kind: Deployment
metadata: 
  name: myapp-blue
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myapp
      version: blue
  template:
    metadata:
      name: myapp
      labels:
        app: myapp
        version: blue
    spec:
      containers:
        - name: myapp
          image: gcr.io/ultra-welder-300122/myapp:blue
          imagePullPolicy: Always
          resources:
            requests:
              cpu: "100m"
              memory: "100Mi"
          ports:
          - containerPort: 8888

My Service yaml

apiVersion: v1
kind: Service
metadata: 
  name: myapp-blue-service
  labels:
    app: myapp
    version: blue
spec:
  type: NodePort
  selector:
    app: myapp
    version: blue
  ports: 
  - protocol: TCP
    port: 80
    targetPort: 8888
---
apiVersion: v1
kind: Service
metadata: 
  name: myapp-green-service
  labels:
    app: myapp
    version: green
spec:
  type: NodePort
  selector:
    app: myapp
    version: green
  ports: 
  - protocol: TCP
    port: 80
    targetPort: 8080

Gke ingress yaml

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: myapp-ingress
  annotations:
    # If the class annotation is not specified it defaults to "gce".
    kubernetes.io/ingress.class: "gce"
spec:
  rules:
  - http:
      paths:
      - path: /*
        backend:
          serviceName: myapp-blue-service
          servicePort: 80
      - path: /green
        backend:
          serviceName: myapp-green-service
          servicePort: 80

Service Status

NAME                  TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
kubernetes            ClusterIP   10.42.0.1      <none>        443/TCP        6h18m
myapp-blue-service    NodePort    10.42.81.106   <none>        80:31664/TCP   4h35m
myapp-green-service   NodePort    10.42.74.168   <none>        80:30246/TCP   4h35m

Ingress Status

kubectl describe ing myapp-ingress
Name:             myapp-ingress
Namespace:        default
Address:          34.95.121.24
Default backend:  default-http-backend:80 (10.41.0.9:8080)
Rules:
  Host        Path  Backends
  ----        ----  --------
  *
              /*        myapp-blue-service:80 (10.41.0.24:8888,10.41.1.16:8888)
              /green    myapp-green-service:80 (10.41.0.27:8080,10.41.1.19:8080)
Annotations:  ingress.kubernetes.io/backends:
                {"k8s-be-30192--a4913825f2ae16d4":"HEALTHY","k8s-be-30246--a4913825f2ae16d4":"HEALTHY","k8s-be-31664--a4913825f2ae16d4":"HEALTHY"}
              ingress.kubernetes.io/forwarding-rule: k8s2-fr-dkti1gqp-default-myapp-ingress-yd16xk65
              ingress.kubernetes.io/target-proxy: k8s2-tp-dkti1gqp-default-myapp-ingress-yd16xk65
              ingress.kubernetes.io/url-map: k8s2-um-dkti1gqp-default-myapp-ingress-yd16xk65
              kubernetes.io/ingress.class: gce

Deployment and pods Status

NAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/myapp-blue                    2/2     2            2           4h18m
deployment.apps/myapp-green                   2/2     2            2           38m
NAME                                               READY   STATUS    RESTARTS   AGE
pod/myapp-blue-9c9cbf5b-c7v7c                      1/1     Running   0          4h18m
pod/myapp-blue-9c9cbf5b-cpvgl                      1/1     Running   0          4h18m
pod/myapp-green-7f56cc9496-hnfkz                   1/1     Running   0          38m
pod/myapp-green-7f56cc9496-v8bb6                   1/1     Running   0          38m

Service status

NAME                  TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
kubernetes            ClusterIP   10.42.0.1      <none>        443/TCP        6h24m
myapp-blue-service    NodePort    10.42.81.106   <none>        80:31664/TCP   4h40m
myapp-green-service   NodePort    10.42.74.168   <none>        80:30246/TCP   4h40m

can you try to access : http://34.95.121.24 and http://34.95.121.24/green — Thanh Nguyen Van
I accessed the external ip provided by the http load balancer. 34.95.121.24 and 34.95.121.24/green. I get a 404 for all redirect path, — Ramesh Rajan
I see that you have configured "/*" instead of "/blue" in "path:" section in the ingress yaml, may it be mistake? — BraveAdmin
/* It's the default backend and works correctly. Any time, i add a path /blue or /green it fails. — Ramesh Rajan

Ramesh Rajan Ramesh Rajan · Accepted Answer · 2021-01-17T19:47:16

I found the problem is with gke ingress controller. Gke ingress controller doesn't provide rewrite-target to /. I am serving my green and blue web pages from the webroot. GKE ingress controller is forwarding the requested URL http://34.95.121.24/green to http://{backend }/green. There is no page hosted on http://{backend }/green as my default page is hosted on http://{backend}/.

There are two solutions to the problem.

Use nginx-ingress controller with annotation to rewrite-target set to /.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /*
        backend:
          serviceName: myapp-blue-service
          servicePort: 80
      - path: /green
        backend:
          serviceName: myapp-green-service
          servicePort: 80

Handle rewrite or redirect non-existing pages in the docker container. In my case it was a nginx webserver.

I created a default server configuration section to route any traffic hitting /green to index.html.

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        root /var/www/html;
        server_name _;
        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                index index.html
                try_files $uri $uri/ =404;
        }
        location /green {
                try_files $uri $uri/ /index.html;
        }

I copied default site configuration to /etc/nginx/sites-avaiable in my docker configuration file.

My sample Dockerfile

FROM ubuntu
RUN apt-get update
RUN apt-get install nginx -y
COPY index.html /var/www/html/
COPY default /etc/nginx/sites-available/
EXPOSE 80
CMD ["nginx","-g","daemon off;"]

gke ingress unable to route traffic to services

1 Answers