How to get kubernetes cluster wide metric

1
votes

I'm running multiple clusters of Kubernetes in GKE. I'm using a heapster + influxdb to get metrics on pods performances. What is the recommended way to get and store cluster wide metrics like:

  • number of pods in namespace
  • total memory requested/limits per namespace
  • total cpu requested/limits per namespace
  • total memory requested/limits per node
  • total cpu requested/limits per node
  • number of pod per state (running, pending, etc)

AFAIK this is out of the scope of heapster but we need to keep an eye on this to understand the state of the cluster.

Thanks!

1
kubernetesgoogle-kubernetes-engine

1 Answers

1
votes

4 of the metrics you mentioned can be visualized using Grafana, if you create custom dashboards and make good use of Grafana templates:

  • total memory requested/limits per namespace
  • total cpu requested/limits per namespace

Sort by "pod_namespace" and aggregate by "cpu/usage" or "cpu/limit" or "cpu/request"

  • total memory requested/limits per node
  • total cpu requested/limits per node

Sort by "hostname" and aggregate by "memory/limit" or "memory/request" or "cpu/usage" or "cpu/limit" or "cpu/request"

See: Heapster storage schema . As soon as you understand the InfluxDB schema you can do magic.

The other 2 are not collected by Heapster, it means you have to come up with you own queries on the Kubernetes API if you want to push that data somewhere, probably also InfluxDB in your case.