kubernetes error : skipping pod synchronization

2
votes

i have configured kubernetes master on centos 7 and kubernetes node on another node centos 7

services running on kube master:

  • kube-controller-manager
  • kube-apiserver
  • kube-scheduler
  • etcd
  • flanneld

service running on kube node:

  • flanneld
  • docker
  • kube-proxy
  • kubelet

all services are up and running and i could see the api url successfully getting all endpoints. http://kube-master:8080 however, when i am running command kube get nodes , getting following error :

skipping pod synchronization. container runtime is down

I am not getting what this error means and how to resolve this. Please suggest.

2
dockerkuberneteskubernetes-pod
Did you find a solution? Seeing the same error with kubeadm 1.10.0 cluster. Work node status NotReady. kubelet log shows a lot of kubelet.go:1794] skipping pod synchronization - [container runtime is down] – DarVar
@DarVar, sometimes docker hangs on deleting a stopped container and then in turn seems to make kubelet hang and fail like this. – Jan Hudec

2 Answers

1
votes

kubelet is the only component with a dependency on the container runtime (Docker in your case). If I were you I'd start investigating the kubelet logs and search for references to Docker. Maybe the user configured to run kubelet doesn't have the necessary permissions to interact with the Docker socket at /var/run/docker.sock.

The content of your logs may help if you need further help.

1
votes

I've seen this problem when docker got into some broken state where it was not able to remove a (specific) stopped container and was leaking zombie processes. Had to power-cycle the node in the end.

CentoOS 7 here too, still at Kubernetes 1.10.0 and Docker CE 18.03.