Pull pod metrics to external prometheus

1
votes

I'm trying to aggregate all the metrics, EC2 (CPU, ram, disk) and Pods (CPU, ram, disk) from inside a EKS cluster using an existing Prometheus (external to the cluster). I started adding data using node-exporter, kube-state-metrics but I'm stuck at deploying metrics-server. Using helm I conf and installed it, kube top and kubectl get --raw "/apis/metrics.k8s.io/v1beta1/pods" | jq are pulling data, but can someone advise on how can I pull all this to my external Prometheus ? Is there a way to expose this to a /metrics URL? Thank you!

1
kubernetesprometheusamazon-eks

1 Answers

1
votes

I think your best bet here is to use prometheus federation. In that setup,

  1. You'll have one in-cluster prometheus and your original external prometheus. The in-cluster prometheus does the actual collection and temporary storage.

  2. In your external prometheus, you'll scrape the /federate endpoint of your in-cluster prometheus. The /federate is specifically designed for the use case you are trying

This is the best method for having performant and reliable scraping of metrics. Getting the metrics via apiserver would not be a good way. Even if it's possible, would put unnecessary strain on the apiserver.