HPA with Custom Metrics

Question

i came across two different approaches to scale on a specific metric and i wonder what is the difference and if there is such in my case.

i have a deployment on GKE that includes scraping and exporting a specific metric from the application to stackdriver. using prometheus-to-sd sidecar. the metric appears on the stackdriver as custom.googleapis.com/dummy/foo

now, usually when i do HPA for custom metric i use it like the following:

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: custom-metric-prometheus-sd
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1beta1
    kind: Deployment
    name: custom-metric-prometheus-sd
  minReplicas: 1
  maxReplicas: 5
  metrics:
  - type: External
    external:
      metricName: "custom.googleapis.com|dummy|foo"
      targetAverageValue: 20

now, the same hpa works also using Pod metrics approach. like:

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: custom-metric-prometheus-sd
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1beta1
    kind: Deployment
    name: custom-metric-prometheus-sd
  minReplicas: 1
  maxReplicas: 5
  metrics:
  - type: Pods
    pods:
      metricName: "custom.googleapis.com|dummy|foo"
      targetAverageValue: 20

it works the same. i understand that when using Pod Metrics HPA will fetch the metrics from all pods and will calculate an average which will be compared to the target value to decide replica counts. its basically the same as if using the targetAverageValue on the External metric. so, in my case both will do basically the same, right? any different maybe in aspects of performance, latency, anything else ?

thanks Chen

Dawid Kruk Dawid Kruk · Accepted Answer · 2020-05-07T15:47:07

Yes, you are pretty much right. The metrics that you have shown in your question will work the same.

Going a little deeper and reading the documentation about metrics type: Kubernetes.io: Docs: Reference: Kubernetes API: v1.18: metricspec-v2beta1-autoscaling

ExternalMetricSource - external refers to a global metric that is not associated with any Kubernetes object. It allows autoscaling based on information coming from components running outside of cluster (for example length of queue in cloud messaging service, or QPS from loadbalancer running outside of cluster).

PodsMetricSource - pods refers to a metric describing each pod in the current scale target (for example, transactions-processed-per-second). The values will be averaged together before being compared to the target value.

Please take in mind that this reference is for api version 1.18. Please use version appropriate to your use case.

The documentation shows that they are designed as different metrics but they can be used as you did.

Apart from the description, one more thing differentiating them is that metric:

External has fields:
- metricName
- metricSelector
- targetAverageValue
- targetValue
Pods has fields:
- metricName
- metricSelector
- targetAverageValue

As you can see above, the Pods metric has no targetValue field.

Each type of metric has it's own set of properties and some of them give the same results when used.

Please let me know if you have any questions in that.

HPA with Custom Metrics

1 Answers