0
votes

I’ve already seen this question; AFAIK I’m doing everything in the answers there.

Using GKE, I’ve deployed a GCP HTTP(S) load balancer-based ingress for a kubernetes cluster containing two almost identical deployments: production and development instances of the same application.

I set up a dedicated port on each pod template to use for health checks by the load balancer so that they are not impacted by redirects from the root path on the primary HTTP port. However, the health checks are consistently failing.

From these docs I added a readinessProbe parameter to my deployments, which the load balancer seems to be ignoring completely.

I’ve verified that the server on :p-ready (9292; the dedicated health check port) is running correctly using the following (in separate terminals):

➜ kubectl port-forward deployment/d-an-server p-ready
➜ curl http://localhost:9292/ -D -
HTTP/1.1 200 OK
content-length: 0
date: Wed, 26 Feb 2020 01:21:55 GMT

What have I missed?

A couple notes on the below configs:

  • The ${...} variables below are filled by the build script as part of deployment.
  • The second service (s-an-server-dev) is almost an exact duplicate of the first (with it’s own deployment) just with -dev suffixes on the names and labels.

Deployment

apiVersion: "apps/v1"
kind: "Deployment"
metadata:
  name: "d-an-server"
  namespace: "default"
  labels:
    app: "a-an-server"
spec:
  replicas: 1
  selector:
    matchLabels:
      app: "a-an-server"
  template:
    metadata:
      labels:
        app: "a-an-server"
    spec:
      containers:
        - name: "c-an-server-app"
          image: "gcr.io/${PROJECT_ID}/an-server-app:${SHORT_SHA}"
          ports:
            - name: "p-http"
              containerPort: 8080
            - name: "p-ready"
              containerPort: 9292
          readinessProbe:
            httpGet:
              path: "/"
              port: "p-ready"
            initialDelaySeconds: 30

Service

apiVersion: "v1"
kind: "Service"
metadata:
  name: "s-an-server"
  namespace: "default"
spec:
  ports:
    - port: 8080
      targetPort: "p-http"
      protocol: "TCP"
      name: "sp-http"
  selector:
    app: "a-an-server"
  type: "NodePort"

Ingress

apiVersion: "networking.k8s.io/v1beta1"
kind: "Ingress"
metadata:
  name: "primary-ingress"
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "primary-static-ipv4"
    networking.gke.io/managed-certificates: "appname-production-cert,appname-development-cert"
spec:
  rules:
    - host: "appname.example.com"
      http:
        paths:
          - backend:
              serviceName: "s-an-server"
              servicePort: "sp-http"
    - host: "dev.appname.example.com"
      http:
        paths:
          - backend:
              serviceName: "s-an-server-dev"
              servicePort: "sp-http-dev"
1

1 Answers

2
votes

I think what's happening here is GKE ingress is not at all informed of port 9292. You are referring sp-http in the ingress which refers to port 8080.

You need to make sure of below:

1.The service's targetPort field must point to the pod port's containerPort value or name.

2.The readiness probe must be exposed on the port matching the servicePort specified in the Ingress.