Kubernetes : Understanding Nodes resources usage, How to handle

0
votes

I'm very new to Kubernetes, I just deployed few services on 2 node cluster.

Following are metrics

Nodes usage

kubectl -n services-namespace top nodes
NAME                                                  CPU(cores)   CPU%      MEMORY(bytes)   MEMORY%   
gke-kubernetes-cluster-n-default-pool-aaaaaaaa-lvzm   151m         16%       1927Mi          72%       
gke-kubernetes-cluster-n-default-pool-bbbbbbbb-6g58   266m         28%       2936Mi          110%

Pods Usage

kubectl -n services-namespace top pods
NAME                                      CPU(cores)   MEMORY(bytes)   
devv-aaa-service-2543400000-7ffbl         5m           269Mi           
devv-bbb-service-2750127145-jfvw4         26m          276Mi           
devv-aaa-service-2543400800-vlgmr         20m          285Mi           
devv-aaa-service-2543000000-g82p1         4m           280Mi           
devv-ccc-service-3204123401-7vv67         19m          283Mi           
devv-aaa-service-2541023480-sd9cm         3m           284Mi           
devv-ccc-service-3207045691-kjlfn         21m          268Mi           
devv-aaa-service-2541204800-64q67         30m          270Mi

What does it mean by MEMORY(%), is it current usage of Node. If so why nodes are taking lot of memory even though pods are not taking full Node memory. Cumulative PODS memory usage is around 2500Mi.

How to handle these Memory and CPU ?

What are the best practices to consider while using clusters.

1
kuberneteskubernetes-health-check

1 Answers

0
votes

Besides running pods , kubernetes nodes also runs OS systems daemons(like udev, sshd) that is required for kubernetes to function . For this the compute resources is reserved which is called as "system-reserved". Also, some more compute resources are reserved for kubernetes system daemons like container runtime and kubelet which is called as "kube-reserved".

So, Node Capacity=kube reserved+system reserved+eviction threshold+Allocatable(available for pods)

You can use "kubectl describe node and in the output look for "System Info" section which can give you detailed info on memory request and limits Also, please go over this link from kubernetes docs: https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/