Kubernetes: What kinds of system aspects should I inspect in the endpoint that serves my readiness probe

Question

I have a simple SpringBoot application (really a REST-based microservice) that I am deploying in Kubernetes.

It has one downstream dependency (another REST-based web service). I know that for the REST endpoint feeding the liveness probe I should not return a failure if my downstream dependency is unavailable / inaccessible (as Kubernetes restarting my microservice pod won't fix the dependency!).

But in the REST endpoint feeding my readiness probe should I be checking downstream dependencies? I'd rather just do something basic but if I need to be checking more then I will.

@RequestMapping("/health")
public String getHealth() {
    return "OK";
}

Laksitha Ranasingha Laksitha Ranasingha · Accepted Answer · 2019-02-06T14:58:58

Assuming, the liveness of your spring-boot app (user's perspective) does not require the dependent service to be up, your idea of checking the status of the Readiness Probe is a right thing to do.

As the dependent app is a REST service, you could expose an HTTP/HTTPS endpoint to be checked by the readiness probe. And keep spring-boot app's health check (or similar) endpoint for liveness probe.

However, beware that your pod which runs first microservice (spring-boot app) could become unresponsive if the dependent service didn't respond.

Therefore, providing correct timeouts (initialDelays & periodDelay) with success and failure thresholds help you mitigate such unresponsive status. For example;

readinessProbe:
  httpGet: # make an HTTP request to dependent's health/readiness endpoint
    port: <port>
    path: /health
    scheme: HTTP 
  initialDelaySeconds: 10 # how long to wait before checking
  periodSeconds: 10 # how long to wait between checks
  successThreshold: 1 # how many successes to hit before accepting
  failureThreshold: 3 # how many failures to accept before failing
  timeoutSeconds: 15

The official doc: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#define-readiness-probes

A good article: https://itnext.io/kubernetes-readiness-probe-83f8a06d33d3

Kubernetes: What kinds of system aspects should I inspect in the endpoint that serves my readiness probe

2 Answers