GlusterFS deployment on k8s cluster-- Readiness probe failed: /usr/local/bin/status-probe.sh

Question

Setup: GlusterFS on k8s(AKS) cluster;

Having issue while glusterfs deploy using gk-deploy; below shared few error snippets please suggest workaround to get my deployment going- thanks in advance.

Getting same error : Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Unhealthy 8m44s (x459 over 3h45m) kubelet, aks-nodepool1-70391060-0 Readiness probe failed: /usr/local/bin/status-probe.sh failed check: systemctl -q is-active gluster-blockd.service Warning Unhealthy 4m2s (x462 over 3h46m) kubelet, aks-nodepool1-70391060-0 Liveness probe failed: /usr/local/bin/status-probe.sh failed check: systemctl -q is-active gluster-blockd.service

When i inspect the pod with the glusterd status - it looks successful, what might be causing the reporting on readiness failed - any ideas / thoughts?

glsuterd status

Answer 1

The probe does not only check for the status of the glusterd.service. There are a few other things that are taken into account. The status-probe.sh script is available from github. The message

Liveness probe failed: /usr/local/bin/status-probe.sh failed check: systemctl -q is-active gluster-blockd.service

indicates that gluster-blockd.service is not running correctly. This can have different causes, you would need to inspect the gluster-blockd.log and tcmu-runner.log to find out what the problem is.

gluster-blockd.service is optional. If you do not need PersistentVolumes with type=block in your environment, you can disable the check.

In the specification of the daemonset, you can change the environment variable GLUSTER_BLOCKD_STATUS_PROBE_ENABLE and set the value to "0".