Liveness-Probe of one pod via another

Question

On my Kubernetes Setup, I have 2 pods - A (via deployment) and B(via DS). Pod B is somehow dependent on Pod A being fully started through. I would now like to set an HTTP Liveness-Probe in Pods B, to restart POD B if health check via POD A fails. Restarting works fine if I put the External IP of my POD A's service in the host. The issue is in resolving DNS name in the host.

It works if I set it like this:

livenessProbe:
  httpGet:
    host: <POD_A_SERVICE_EXTERNAL_IP_HERE>
    path: /health
    port: 8000

Fails if I set it like this:

livenessProbe:
  httpGet:
    host: auth
    path: /health
    port: 8000

Failed with following error message:

Liveness probe failed: Get http://auth:8000/health: dial tcp: lookup auth on 8.8.8.8:53: no such host

ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

Is the following line on the above page true for HTTP Probes as well? "you can not use a service name in the host parameter since the kubelet is unable to resolve it."

Rico Rico · Accepted Answer · 2020-07-25T02:27:59

Correct 👍, DNS doesn't work for liveness probes, the kubelet network space cannot basically resolve any in-cluster DNS.

You can consider putting both of your services in a single pod as sidecars. This way they would share the same address space if one container fails then the whole pod is restarted.

Another option is to create an operator 🔧 for your pods/application and basically have it check the liveness through the in-cluster DNS for both pods separately and restart the pods through the Kubernetes API.

You can also just create your own script in a pod that just calls curl to check for a 200 OK and kubectl to restart your pod if you get something else.

Note that for the 2 options above you need to make sure that Coredns is stable and solid otherwise your health checks might fail to make your services have potential downtime.

✌️☮️

Liveness-Probe of one pod via another

1 Answers