Kubernetes Ingress get Unhealthy backend services on Google Kubernetes Engine

Question

I'm trying to deploy two services on Google container engine, I have created a cluster with 3 Nodes. My docker images are in private docker hub repo that's why I have created a secret and used in Deployments, The ingress is creating a load balancer in the Google cloud console but it shows that backend services are not healthy and inside the kubernetes section under workloads it says Does not have minimum availability.

I'm new to kubernetes, what can be a problem?

Here are my yamls:

Deployment.yaml:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: pythonaryapp
  labels:
    app: pythonaryapp
spec:
  replicas: 1 #We always want more than 1 replica for HA
  selector:
    matchLabels:
      app: pythonaryapp
  template:
    metadata:
      labels:
        app: pythonaryapp
    spec:
      containers:
      - name: pythonaryapp #1st container
        image: docker.io/arycloud/docker_web_app:pythonaryapp #Dockerhub image
        ports:
        - containerPort: 8080 #Exposes the port 8080 of the container
        env:
        - name: PORT #Env variable key passed to container that is read by app
          value: "8080" # Value of the env port.
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8080
          periodSeconds: 2
          timeoutSeconds: 2
          successThreshold: 2
          failureThreshold: 10
      imagePullSecrets:
         - name: docksecret
---

kind: Deployment
apiVersion: apps/v1
metadata:
  name: pythonaryapp1
  labels:
    app: pythonaryapp1
spec:
  replicas: 1 #We always want more than 1 replica for HA
  selector:
    matchLabels:
      app: pythonaryapp1
  template:
    metadata:
      labels:
        app: pythonaryapp1
    spec:
      containers:
      - name: pythonaryapp1 #1st container
        image: docker.io/arycloud/docker_web_app:pythonaryapp1 #Dockerhub image
        ports:
        - containerPort: 8080 #Exposes the port 8080 of the container
        env:
        - name: PORT #Env variable key passed to container that is read by app
          value: "8080" # Value of the env port.
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8080
          periodSeconds: 2
          timeoutSeconds: 2
          successThreshold: 2
          failureThreshold: 10
      imagePullSecrets:
         - name: docksecret
---

And here's services.yaml:

kind: Service
apiVersion: v1
metadata:
  name: pythonaryapp
spec:
  type: NodePort
  selector:
    app: pythonaryapp
  ports:
  - protocol: TCP
    port: 8080
---

---
kind: Service
apiVersion: v1
metadata:
  name: pythonaryapp1
spec:
  type: NodePort
  selector:
    app: pythonaryapp1
  ports:
  - protocol: TCP
    port: 8080
---

And Here's my ingress.yaml:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: mysvcs
spec:
  rules:
  - http:
      paths:
      - path: /
        backend:
          serviceName: pythonaryapp
          servicePort: 8080
      - path: /<name>
        backend:
          serviceName: pythonaryapp1
          servicePort: 8080

Update:

Here's flask service code:

from flask import Flask

app = Flask(__name__)


@app.route('/')
def hello_world():
    return 'Hello World, from Python Service.', 200




if __name__ == '__main__':
    app.run()

And, on running the container of it's docker image it retunrs 200 sttaus code at the root path /.

Thanks in advance!

Can you make sure the service is returning a 200 response code? The L7LB healthcheck will resolve against the service. use curl [node_external_ip]:[nodeport] and make sure you get a 200 response — Patrick W
you can get the node external IP with "gcloud compute instances list", pick any node. You can get the nodeport IP using "kubectl get svc" and look for the port (it will be in the 30000 range) — Patrick W
Make sure the firewall rules allow the curl, if you are using the default network, the best way to test this is doing so from a VM within your project and use the node's internal IP address — Patrick W
I'm asking the above because of the "Does not have minimum availability" which implies an issue with pod deployment. Can you also provide the output of "kubectl get po" — Patrick W
Hi @PatrickW, Actually one service is working but another is not, can you take a look at this question: stackoverflow.com/q/52685701/7644562 , please! — Abdul Rehman

Notauser Notauser · Accepted Answer · 2018-10-04T03:23:15

Have a look at this post. It might contain helpful tips for your issue. For example I do see a readiness probe but not a liveness probe in your config files.

This post suggests that “Does not have minimum availability” in k8s could be a result of a CrashloopBackoff caused by a failing liveness probe.

Kubernetes Ingress get Unhealthy backend services on Google Kubernetes Engine

2 Answers