Horizontal Pod autoscaling

You can automatically adjust the number of Pods in your Google Kubernetes Engine (GKE) workloads based on CPU or memory usage, custom metrics, or external metrics by using horizontal Pod autoscaling. Scaling Pod replicas to match changing demand helps to maintain consistent application performance and release unused Pod resources when demand drops.

Horizontal Pod autoscaling doesn't change the number of nodes in a GKE cluster. To automatically scale cluster nodes when Pod counts change, see About GKE cluster autoscaling. To set up Pod scaling on your workloads, see Configuring horizontal Pod autoscaling.

This document is for developers and operators who configure and manage workload scaling. For more information about common roles and example tasks that we reference in Google Cloud content, see Common GKE user roles and tasks.

Supported scaling metrics

A workload's resource requirements can change over time depending on traffic patterns and external dependencies. A HorizontalPodAutoscaler can scale the number of Pods in your workload based on one or more of the following metric types:

  • Actual resource usage: the autoscaler scales Pods when CPU or memory usage exceeds a target threshold, expressed as a raw value or a percentage of the Pod's resource request.
  • Custom metrics: the autoscaler scales Pods based on any metric reported by a Kubernetes object in a cluster, such as client requests or I/O writes per second. Custom metrics are useful when your application experiences network or I/O bottlenecks rather than CPU or memory constraints.
  • External metrics: the autoscaler scales Pods based on a metric from an application or service outside your cluster. For example, when a workload ingests Pub/Sub requests, you can configure a HorizontalPodAutoscaler to add Pods above a queue threshold and remove them as the queue shrinks.

You can also combine horizontal Pod autoscaling with vertical Pod autoscaling. For more information about compatibility rules, see the limitations section of this document.

How horizontal Pod autoscaling works

Each configured horizontal Pod autoscaler operates using a control loop. A separate horizontal Pod autoscaler exists for each workload. Each horizontal Pod autoscaler periodically checks a given workload's metrics against the target thresholds you configure, and changes the shape of the workload automatically.

Per-Pod resources

For resources that are allocated per-Pod, such as CPU, the controller queries the resource metrics API for each container running in the Pod.

  • If you specify a raw value for CPU or memory, the value is used.
  • If you specify a percentage value for CPU or memory, the horizontal Pod autoscaler calculates the average utilization value as a percentage of that Pod's CPU or memory requests.
  • Custom and external metrics are expressed as raw values or average values.

The controller uses the average or raw value for a reported metric to produce a ratio, and uses that ratio to autoscale the workload. You can read a description of the horizontal Pod autoscaler algorithm in the Kubernetes project documentation.

Responding to multiple metrics

If you configure a workload to autoscale based on multiple metrics, the horizontal Pod autoscaler evaluates each metric separately and uses the scaling algorithm to determine the new workload scale based on each one. The largest scale is selected for the autoscale action.

If one or more of the metrics are unavailable for some reason, the horizontal Pod autoscaler still scales up based on the largest size calculated, but does not scale down.

Preventing thrashing

Thrashing refers to a situation in which the horizontal Pod autoscaler attempts to perform subsequent autoscaling actions before the workload finishes responding to prior autoscaling actions. To prevent thrashing, the horizontal Pod autoscaler chooses the largest recommendation from a specified stabilization window. This behavior is controlled by the scaleDown.stabilizationWindowSeconds field in the HPA behavior specification, which defaults to 300 seconds (five minutes) in GKE.

To prevent minor fluctuations from causing unnecessary scaling events, the horizontal Pod autoscaler uses a tolerance value of 10%. For a given target, no scaling action is taken if the ratio of the current metric value to the target metric value is within 10% of 1.0.

Scaling to and from zero

In GKE version 1.37 or later, you can configure the horizontal Pod autoscaler to scale a workload down to 0 replicas when there is no demand, and automatically scale the workload back up when demand resumes. Scaling to and from zero includes the following behaviors:

  • Tolerance bypass: when a workload is at 0 replicas, GKE bypasses the standard metric tolerance check (0.9 to 1.1 ratio). Any metric value greater than zero, such as a single message in a queue, immediately triggers a scale-up to at least 1 replica without waiting to cross a tolerance threshold.
  • Stabilization windows: scale-up defaults to 0 seconds (scaleUp.stabilizationWindowSeconds: 0) for immediate activation, while scale-down defaults to 300 seconds (scaleDown.stabilizationWindowSeconds: 300) to help ensure five continuous minutes of zero demand before scaling to 0 replicas.
  • Status conditions: when scaled to 0 replicas by the HPA, the controller reports the ScaledToZero: True condition while remaining active (ScalingActive: True). If a workload is manually scaled to 0 replicas outside the HPA (such as with the kubectl scale command), autoscaling pauses (ScalingActive: False) until the workload is scaled back to 1 or more replicas.

For step-by-step instructions, see Scale GKE workloads to and from zero using HPA.

Limitations

  • Unless you use HPA rightsizing with VPA (Preview), don't use the horizontal Pod autoscaler together with standard vertical Pod autoscaler on CPU or memory. You can use the horizontal Pod autoscaler with the vertical Pod autoscaler for other metrics, or configure multidimensional Pod autoscaling to scale horizontally on CPU and vertically on memory at the same time.
  • If you have a Deployment, don't configure horizontal Pod autoscaling on the ReplicaSet or Replication Controller backing it. When you perform a rolling update on the Deployment or Replication Controller, it is replaced by a new Replication Controller. Instead configure horizontal Pod autoscaling on the Deployment itself.
  • You can't use horizontal Pod autoscaling for workloads that cannot be scaled, such as DaemonSets.
  • Horizontal Pod autoscaling exposes metrics as Kubernetes resources, which imposes limitations on metric names such as no uppercase or '/' characters. Your metric adapter might allow renaming. For example, see the prometheus-adapter as operator.
  • The horizontal Pod autoscaler doesn't scale down if any of the metrics that it's configured to monitor are unavailable. To check if you have unavailable metrics, see Viewing details about a horizontal Pod autoscaler.

Scalability

While the horizontal Pod autoscaler doesn't have a hard limit on the number of supported HPA objects, its performance can be affected as this number grows. Specifically, the period between HPA recalculations might become longer than the standard 15 seconds.

  • In GKE minor version 1.31 or 1.32, if the Performance HPA profile is configured, the recalculation period should stay within 15 seconds with up to 1,000 HPA objects. Learn how to configure the Performance HPA profile.
  • In GKE minor version 1.33 or later, if the Performance HPA profile is configured, the recalculation period should stay within 15 seconds with up to 5,000 HPA objects. The Performance HPA profile is enabled by default on all clusters that meet the requirements.
  • If the Performance HPA profile is not configured, the recalculation period should stay within 15 seconds with up to 300 HPA objects.

The following factors can also affect performance:

  • Scaling on multiple metrics: each metric adds a fetch call for recommendation calculations, which affects the recalculation period.
  • The latency of the custom metrics stack: response times over approximately 50 milliseconds would be more than typically observed with the standard Kubernetes metrics, affecting the recalculation period.

Interacting with HorizontalPodAutoscaler objects

You can configure a horizontal Pod autoscaler for a workload, and get information about autoscaling events and what caused them, by visiting the Workloads page in the Google Cloud console.

Each horizontal Pod autoscaler exists in the cluster as a HorizontalPodAutoscaler object. You can use commands like kubectl get hpa or kubectl describe hpa HPA_NAME to interact with these objects.

You can also create HorizontalPodAutoscaler objects using the kubectl autoscale command.

What's next