apply HPA for Statefulset in kubernetes?

Question

I am trying to setup HPA for my statefulset(for elasticsearch) in kubernetes environment. I am planning to scale the statefulset using the cpu utilization. I have created the metric server from https://github.com/stefanprodan/k8s-prom-hpa/tree/master/metrics-server.

and my HPA yaml for statefulset is as folows:

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: dz-es-cluster
spec:
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: StatefulSet
    name: dz-es-cluster
  minReplicas: 2
  maxReplicas: 3
  metrics:
  - type: Resource
    resource:
      name: cpu
      targetAverageUtilization: 80

But getting output in hpa as follows:

Conditions:
  Type         Status  Reason          Message
  ----         ------  ------          -------
  AbleToScale  False   FailedGetScale  the HPA controller was unable to get the target's current scale: the server could not find the requested resource
Events:
  Type     Reason          Age                From                       Message
  ----     ------          ----               ----                       -------
  Warning  FailedGetScale  1m (x71 over 36m)  horizontal-pod-autoscaler  the server could not find the requested resource

someone please help me..

@PrafullLadha, these this the output. root@ip:~# kubectl top nodes NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% ip-***-20-32-***.ec2.internal 1454m 9% 18392Mi 28% ip-***-20-34-***.ec2.internal 2130m 13% 31761Mi 49% — manu thankachan
Your metrics server is running fine. Did you specify the resources section in your statefulset. That is required section for HPA to work. kubernetes.io/docs/concepts/configuration/… — Prafull Ladha
There is an entry for resources in my statefulset yaml and CPU limit is set to 1 and request to 0.25. Yes, my metrics server is running fine. I have tested its working by autoscaling one of my Deployment. Issue is only with statefulset. — manu thankachan

Prafull Ladha Prafull Ladha · Accepted Answer · 2019-02-13T07:32:14

The support for autoscaling the statefulsets using HPA is added in kubernetes 1.9, so your version doesn't has support for it. After kubernetes 1.9, you can autoscale your statefulsets using:

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: YOUR_HPA_NAME
spec:
  maxReplicas: 3
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: StatefulSet
    name: YOUR_STATEFUL_SET_NAME
  targetCPUUtilizationPercentage: 80

Please refer the following link for more information:

https://github.com/kubernetes/kubernetes/issues/44033

apply HPA for Statefulset in kubernetes?

1 Answers