GCP Kubernetes Ingress Unhealthy Service failing

Question

I have a deployment, service, and ingress for a Django application in Google Cloud Kubernetes. When I apply all configuration, ingress fails because the Django service is unhealthy, I get 502 gateway when I visit the domain. My Django app does return a status 200 for the endpoint "/"; however, when I swapped my image with another some Google example http server, ingress worked fine. So it seems it could be my image which uses Gunicorn, but I can't seem to solve the issue. See the code examples below. I would appreciate a hand, thank you a lot!

GCP Error

ingress.kubernetes.io/backends: 
{"k8s-be-xxxx":"HEALTHY","k8s-be-xxxxxx":"UNHEALTHY"}

Curl to port-forwarded pod locally

curl -iL http://127.0.0.1:8080/
HTTP/1.1 200 OK
Server: gunicorn/20.0.4
Date: Fri, 06 Nov 2020 03:29:03 GMT
Connection: close
Content-Type: text/html; charset=utf-8
X-Frame-Options: SAMEORIGIN
Content-Length: 45

<html><body><h1>API</h1></body></html>

Django URLS

# urls.py
def index(request):
    html = "<html><body><h1>API</h1></body></html>"
    return HttpResponse(html)

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', index),
]

Dockerfile

FROM python:3.8-slim

ENV APP_HOME /app
WORKDIR $APP_HOME

# Install dependencies.
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy local code to the container image.
COPY ./api .

ENV PYTHONUNBUFFERED TRUE

k8 Deployment

# Deployment 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: api
  name: api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
      name: api
    spec:
      containers:
      - name: api
        image: gcr.io/xxxxx/django-api
        ports:
        - containerPort: 8080
          protocol: TCP
        imagePullPolicy: Always
        command: [ "gunicorn", "--workers", "3", "--bind", ":8080",
                   "--log-level", "INFO", "--timeout", "90", "api.wsgi:application" ]
        resources:
          limits:
            cpu: 50m
            memory: 2048Mi
          requests:
            cpu: 50m
            memory: 2048Mi

k8 Service

apiVersion: v1
kind: Service
metadata:
  labels:
    app: api
  name: api
spec:
  type: NodePort
  selector:
    app: api
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080

k8 Ingress

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: core-ingress
spec:
  rules:
    - host: api.example.com
      http:
        paths:
          - backend:
              serviceName: api
              servicePort: 8080
            path: /*

Could you share both Dockerfiles? The one that is working as well. — Mariusz K.
I shared the Dockerfile in the description, I tested with a simple-http server image from Google examples, and that worked fine. — Avi

Niels Uitterdijk Niels Uitterdijk · Accepted Answer · 2020-12-02T20:00:23

I just ran into the same problem. After some debugging it turned out my problem was the health check. If the url's '/' and '/healthz' do not return a HTTP 200, the Ingress doesn't allow any traffic. So add a simple 'path('/healthz', index)' to your urls did the job for me. Obviously give it some time.

GCP Kubernetes Ingress Unhealthy Service failing

1 Answers