Unable to retrieve Heapster Metric model

Question

I am running my kubernetes v1.6.11-gke.0 cluster on GKE. Within the cluster heapster-v1.3.0 (gcr.io/google_containers/heapster-amd64:v1.3.0) pod is running.

However when trying to retrieve heapster metrics through the REST API, there appears to be an issue, as showing below:

localhost:8001/api/v1/model/metrics
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "the server could not find the requested resource",
  "reason": "NotFound",
  "details": {},
  "code": 404
}

and

localhost:8001/api/v1/proxy/namespaces/kube-system/services/heapster/
404 page not found

Moreover, kube-dns seems unreachable too

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "no endpoints available for service \"kube-dns\"",
  "reason": "ServiceUnavailable",
  "code": 503
}

Using kubectl cluster-info services seem to be working well.

Any suggestion on where to start debugging the issue from?

Updated: results of kubectl get pods -n kube-system

fluentd-gcp-v2.0-jp9qq                               2/2       Running   0          87d
fluentd-gcp-v2.0-s2hpp                               2/2       Running   0          19h
fluentd-gcp-v2.0-xlbq2                               2/2       Running   0          19h
heapster-v1.3.0-1288166888-2j4b2                     2/2       Running   0          19h
kube-dns-323615064-49klg                             3/3       Running   0          19h
kube-dns-autoscaler-2667913178-dv2s3                 1/1       Running   0          19h
kube-proxy-gke-acme-air-default-pool-c005178d-gkq6   1/1       Running   0          19h
kube-proxy-gke-acme-air-default-pool-c005178d-n3l7   1/1       Running   0          19h
kube-proxy-gke-acme-air-default-pool-c005178d-v9nn   1/1       Running   0          19h
kubernetes-dashboard-2917854236-wgh8f                1/1       Running   0          19h
l7-default-backend-1044750973-h5s9g                  1/1       Running   0          19h

results of kubectl get svc -n kube-system

NAME                   CLUSTER-IP     EXTERNAL-IP   PORT(S)         AGE
default-http-backend   10.43.249.84   <nodes>       80:31149/TCP    160d
heapster               10.43.242.96   <none>        80/TCP          160d
kube-dns               10.43.240.10   <none>        53/UDP,53/TCP   160d
kubernetes-dashboard   10.43.243.31   <none>        80/TCP          160d

Could you share output of kubectl get pods -n kube-system and kubectl get svc -n kube-system ? — Droga Mleczna

Khaled Khaled · Accepted Answer · 2017-11-07T17:46:25

Heaspter metrics API is to be accessed through Heapster service, not the main API. the path to model API is http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/heapster/api/v1/model/metrics rather than localhost:8001/api/v1/model/metrics

Unable to retrieve Heapster Metric model

1 Answers