This document provides an overview of how to scale compute capacity and obtain
hardware resources across Google Cloud to maintain performance, control costs,
and avoid downtime. To handle variable demand, you can autoscale
Compute Engine instances, orchestrate containers in Google Kubernetes Engine (GKE)
or workloads in clusters, reserve capacity in advance, and distribute traffic
across resources.

The recommendations in this document apply to all available compute options,
except Cloud Run and Batch, which manage and scale compute
infrastructure for you. For more information about the compute options that you
can use in Google Cloud before you scale, see
[Choose compute options](https://docs.cloud.google.com/docs/compute-area/choose-compute-options.md.txt).

## Compute infrastructure scaling options

To scale your compute capacity and improve resource obtainability, choose from
the features that your compute service supports. You can combine multiple
features---such as reservations, autoscalers, and load balancers---to
match your workload requirements. If you want to evaluate these features side by
side based on their purpose, supported compute options, triggers, and use cases,
then see [Compare scaling options](https://docs.cloud.google.com/docs/compute-area/scale-overview.md.txt#compare) in this document.

Each compute service scales capacity and helps you obtain resources by using the
following options:

- **All compute options**: across compute services except
  Cloud Run for reservations, you can have a high level of
  assurance for hardware availability in advance and route user requests
  across healthy backends. You can use the following cross-product options:

  - **[Reservations](https://docs.cloud.google.com/compute/docs/instances/choose-reservation-type.md.txt)**:
    request capacity in a specific zone for immediate use or for a future
    date. Reservations aren't an autoscaler, but they provide a high level
    of assurance that hardware such as graphics processing units (GPUs),
    tensor processing units (TPUs), and Local SSD disks is available when
    your workloads scale out.

  - **[Traffic distribution with Cloud Load Balancing](https://docs.cloud.google.com/load-balancing/docs/load-balancing-overview.md.txt)**:
    distribute incoming requests across healthy compute instances, node
    pools, or containers, and route traffic to new resources as capacity
    scales.

- **Compute Engine**: you scale capacity when you create virtual
  machine (VM) or bare metal instances, or when you use managed instance
  groups (MIGs). A MIG manages a group of compute instances as a single entity
  and can add or remove capacity based on load, schedules, or hardware
  availability. For workloads that require hardware isolation, you can also
  autoscale sole-tenant node groups on dedicated physical servers. You can use
  the following options:

  - **[MIG autoscaling](https://docs.cloud.google.com/compute/docs/autoscaler.md.txt)**: add or remove compute
    instances in a MIG based on CPU usage, load balancer capacity,
    Cloud Monitoring metrics, schedules, or predictive forecasts.

  - **[MIG instance flexibility](https://docs.cloud.google.com/compute/docs/instance-groups/about-instance-flexibility.md.txt)**:
    rank multiple machine types in a MIG to improve resource obtainability.

  - **[Bulk mode and resize requests in MIGs](https://docs.cloud.google.com/compute/docs/instance-groups/overview-add-vms-all-at-once-in-mig.md.txt)**:
    add all requested compute instances in a MIG at the same time when full
    capacity becomes available, including for GPUs, TPUs, and H4D instances
    with Flex-start VMs.

  - **[Standby pools in MIGs](https://docs.cloud.google.com/compute/docs/instance-groups/accelerate-mig-scale-out-with-standby-pools.md.txt)**:
    keep a standby pool of pre-configured suspended or stopped compute
    instances in a MIG to speed up scale-out. When traffic or CPU usage
    increases, the MIG resumes or starts the compute instances instead of
    creating new ones.

  - **[Bulk compute instance creation](https://docs.cloud.google.com/compute/docs/instances/multiple/about-bulk-creation.md.txt)**:
    create multiple compute instances in a single API request with automatic
    zone and machine type selection.

  - **[Sole-tenant node group autoscaling](https://docs.cloud.google.com/compute/docs/nodes/autoscaling-node-groups.md.txt)**:
    add or remove dedicated physical sole-tenant nodes based on guest
    compute instance demand to maintain hardware isolation and optimize
    bring-your-own-license costs.

  - **[Custom orchestrator](https://docs.cloud.google.com/compute/docs/instances/instance-creation-overview.md.txt)**:
    call the Compute Engine API from your own or open-source schedulers to
    create, resize, and delete compute instances based on custom job queues
    or business rules.

- **Cluster Director**: Google deploys managed clusters for artificial
  intelligence (AI), machine learning (ML), and high performance computing
  (HPC) workloads that use either Slurm orchestration or managed compute
  orchestration backed by zonal MIGs. In Slurm clusters, the Slurm scheduler
  creates dynamic compute nodes when queued jobs exceed static baseline nodes
  and powers them down after five minutes of idle time. In managed compute
  clusters, you run workloads over SSH or integrate your own scheduler, and
  scale capacity when you adjust MIG target sizes. You can use the following
  options:

  - **[Cluster orchestration](https://docs.cloud.google.com/cluster-director/docs/orchestration.md.txt)**:
    autoscale dynamic compute nodes within managed Slurm partitions based on
    job queue demand, or provision managed compute clusters backed by zonal
    MIGs to run workloads over SSH or integrate your own scheduler.

  - **[Live cluster modification](https://docs.cloud.google.com/cluster-director/docs/manage/modify-cluster.md.txt)**:
    update an active cluster to raise static or maximum dynamic node counts,
    resize or add MIGs, add compute resource configurations, or add
    partitions and nodesets without disruption to active jobs.

  - **[Capacity consumption options](https://docs.cloud.google.com/cluster-director/docs/obtain-capacity.md.txt)**:
    obtain cluster compute nodes through future reservations for capacity
    blocks, future reservations in calendar mode, Flex-start VMs,
    Spot VMs, or on-demand compute instances.

- **Cluster Toolkit**: you define self-managed AI, ML, and HPC
  clusters as code with YAML blueprints and Terraform modules orchestrated by
  Slurm or GKE. Capacity scales based on Slurm job queues or
  Kubernetes Pod demand, or when you edit and redeploy your cluster
  blueprint. You can use the following options:

  - **[Slurm dynamic nodesets](https://docs.cloud.google.com/cluster-toolkit/docs/slurm/node-types.md.txt)** :
    configure the `slurm-nodeset-dynamic` module so that Slurm creates and
    deletes ephemeral compute instances based on queued jobs.

  - **[GKE node pool autoscaling](https://docs.cloud.google.com/cluster-toolkit/docs/modules/gke-node-pool.md.txt)** :
    set minimum and maximum node counts in your blueprint with the
    `gke-node-pool` module so that GKE adds or removes nodes
    as Pod demand changes.

  - **[Live blueprint reconfiguration](https://docs.cloud.google.com/cluster-toolkit/docs/slurm/reconfigure-cluster.md.txt)**:
    update and redeploy blueprints to add or resize partitions and static
    nodes in an active Slurm cluster or to resize node pools in a deployed
    GKE cluster.

  - **[Multi-reservation Slurm partitions](https://docs.cloud.google.com/cluster-toolkit/docs/slurm/scale-across-reservations.md.txt)**:
    group multiple reservation-backed nodesets into one Slurm partition so
    that large jobs span capacity across separate reservations.

- **GKE** : capacity scales across two layers. Workload
  autoscalers adjust Pod replica counts or container CPU and memory requests
  as demand changes. Infrastructure autoscalers add or remove underlying
  nodes and node pools. Google manages node capacity for you in
  Autopilot mode, whereas you configure cluster autoscaler, node pool
  auto-creation, and custom `ComputeClass` resources in Standard
  mode. Startup accelerators also reduce the time that new Pods need before
  they serve traffic. You can use the following options:

  - **Node and cluster capacity**: adjust node counts, create node pools,
    and obtain hardware with the following features:

    - **[Custom ComputeClasses](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/about-custom-compute-classes.md.txt)**:
      define prioritized hardware fallback rules, Spot VMs
      or Flex-start preferences, reservation consumption, active
      migration to higher-priority hardware, and workload-level node pool
      auto-creation.

    - **[Cluster autoscaler](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler.md.txt)**:
      add or remove nodes in existing Standard mode node pools
      when Pods wait for resources or when nodes are underutilized.

    - **[Node pool auto-creation](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/node-auto-provisioning.md.txt)**:
      create and delete node pools in parallel based on the CPU, memory,
      GPU, TPU, or zone requirements of pending Pods.

    - **[Flex-start provisioning](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/dws.md.txt)**:
      obtain GPUs, TPUs, and H4D nodes for up to seven days either node
      by node or all at once through queued provisioning, and replace
      expiring nodes through ComputeClasses.

    - **[Capacity buffers](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/capacity-buffer.md.txt)** :
      pre-provision spare active or suspended standby nodes, including
      active buffers that align with `ComputeClass` priorities, so that
      pending Pods schedule without a wait for new nodes.

    - **[Container-optimized compute platform](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview.md.txt#autopilot-compute-platform)** :
      speed up node allocation and Pod placement with `Balanced` and
      `Scale-Out` ComputeClasses for Autopilot clusters and
      Autopilot mode workloads in Standard clusters.

    - **[Sole-tenant nodes in GKE](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/sole-tenancy.md.txt)**:
      run Standard mode node pools on dedicated physical servers
      to meet isolation or license requirements.

  - **Pod and workload capacity**: adjust Pod replicas, container sizes,
    and batch queues with the following features:

    - **[Horizontal Pod autoscaling (HPA)](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/horizontalpodautoscaler.md.txt)**:
      add or remove Pod replicas based on CPU usage, memory usage, or
      custom metrics.

    - **[Vertical Pod autoscaling (VPA)](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/verticalpodautoscaler.md.txt)**:
      adjust container CPU and memory requests and limits based on
      historical usage.

    - **[Multidimensional Pod autoscaling (MPA)](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/multidimensional-pod-autoscaling.md.txt)**:
      combine HPA on CPU and VPA on memory for the same workload at the
      same time.

    - **[Custom and external metrics](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/custom-and-external-metrics.md.txt)**:
      trigger HPA scale decisions from application metrics or external
      services such as Pub/Sub and Cloud Load Balancing.

    - **[Scale to zero](https://docs.cloud.google.com/kubernetes-engine/docs/tutorials/scale-to-from-zero-hpa.md.txt)**:
      scale idle Pods and node pools down to zero replicas and wake them
      when demand resumes in HPA custom and external metrics or in
      Kubernetes Event-driven Autoscaling (KEDA) triggers.

    - **[Kueue job queues](https://docs.cloud.google.com/kubernetes-engine/docs/tutorials/kueue-intro.md.txt)**:
      queue batch and AI workloads, share resource quotas across teams,
      and coordinate simultaneous node allocation.

  - **Startup acceleration**: shorten container initialization and model
    load times during scale-out with the following features:

    - **[Cloud Storage FUSE with parallel streaming](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/cloud-storage-fuse-csi-driver.md.txt)**:
      mount Cloud Storage buckets as local file systems with parallel
      reads and file caches.

    - **[CPU startup boost](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/boost-application-startup.md.txt)**:
      grant temporary extra CPU to containers while they initialize, and
      then return CPU requests to baseline values.

    - **[Image streaming](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/image-streaming.md.txt)**:
      stream container image blocks on demand from Artifact Registry so that
      Pods start without waiting for the entire image download.

    - **[Secondary boot disks](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/data-container-image-preloading.md.txt)**:
      attach preloaded Persistent Disk volumes with container images or model
      data to nodes in parallel during node boot.

    - **[In-place Pod resize](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/verticalpodautoscaler.md.txt#inplaceorrecreate_mode)**:
      update CPU and memory allocations on active containers without Pod
      restarts.

    - **[Pod snapshots](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/pod-snapshots.md.txt)**:
      restore Pod replicas from saved memory and file system checkpoints
      instead of a cold boot.

    - **[Run:ai Model Streamer](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/run-ai-model-streamer.md.txt)** :
      stream model weights in `safetensors` format from
      Cloud Storage or persistent volumes in parallel into GPU
      or TPU memory.

    - **[Volume Populator](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/about-volume-populator.md.txt)**:
      preload Google Cloud Hyperdisk ML volumes with model weights and
      datasets from Cloud Storage before Pods start.

- **Google Cloud VMware Engine** : Google manages dedicated VMware private cloud
  clusters where capacity expands or shrinks as physical ESXi hosts join or
  leave the cluster. To add or remove ESXi hosts when sustained CPU, memory,
  or vSAN storage usage crosses your configured thresholds, use
  [autoscale policies](https://docs.cloud.google.com/vmware-engine/docs/howto-autoscale.md.txt).

## Compare scaling options

To select the scaling or resource obtainability option that matches your
operational requirements, compare features in the following table:
Reservations Traffic distribution with Cloud Load Balancing MIG autoscaling MIG instance flexibility Bulk mode and resize requests in MIGs Standby pools in MIGs Bulk compute instance creation Sole-tenant node group autoscaling Custom orchestrator Slurm orchestration and dynamic nodes Live cluster modification Live blueprint reconfiguration Custom ComputeClasses Cluster autoscaler Node pool auto-creation Flex-start provisioning in GKE Capacity buffers Container-optimized compute platform HPA VPA MPA Custom and external metrics Scale to zero Kueue job queues Cloud Storage FUSE with parallel streaming CPU startup boost Image streaming Secondary boot disks In-place Pod resize Pod snapshots Run:ai Model Streamer Volume Populator Autoscale policies <button class="clear-all">Clear all</button>

| Scaling option | Primary purpose | Supported compute options | Compatible scaling options | Scaling trigger | Use cases |
|---|---|---|---|---|---|
| **[Reservations](https://docs.cloud.google.com/compute/docs/instances/choose-reservation-type.md.txt)** | Reserve zone capacity for immediate use or for a future date with a high level of availability assurance | All compute options, except Cloud Run | All scaling options | Manual or calendar reservation request; not an autoscaler | - Planned growth and peak scale events - Large migrations - Backup and disaster recovery - High-demand resources, such as GPUs and TPUs |
| **[Traffic distribution with Cloud Load Balancing](https://docs.cloud.google.com/load-balancing/docs/load-balancing-overview.md.txt)** | Distribute incoming traffic across healthy compute instances, node pools, or containers | All compute options | All scaling options | Incoming user requests and backend health checks | - High-traffic web applications and APIs - Multi-zone and multi-region availability - Zero-downtime scale-out |
| **[MIG autoscaling](https://docs.cloud.google.com/compute/docs/autoscaler.md.txt)** | Add or remove compute instances in a MIG for variable workload demand | Compute Engine | Reservations, MIG instance flexibility, standby pools in MIGs, sole-tenant node group autoscaling, custom orchestrator | CPU usage, load balancer capacity, Cloud Monitoring metrics, schedules, or predictive forecasts | - Web services and APIs - Parallel batch workloads - Multi-zone high availability - Stateless microservices |
| **[MIG instance flexibility](https://docs.cloud.google.com/compute/docs/instance-groups/about-instance-flexibility.md.txt)** | Rank multiple machine types in a MIG to improve resource obtainability | Compute Engine | MIG autoscaling, reservations, Spot VMs | When primary machine types lack capacity during MIG instance creation or scale-out | - Higher compute instance obtainability - Cost and performance balance across machine series - Stateless web and batch workloads |
| **[Bulk mode and resize requests in MIGs](https://docs.cloud.google.com/compute/docs/instance-groups/overview-add-vms-all-at-once-in-mig.md.txt)** | Add all requested compute instances in a MIG at the same time when full capacity becomes available | Compute Engine and Cluster Director | Reservations, live cluster modification, custom orchestrator | Target size set in bulk mode or creation of a resize request, fulfilled when full hardware capacity is available | - Distributed AI/ML model training - Multi-node jobs that require all compute instances at the same time - Time-flexible GPU, TPU, and H4D workloads |
| **[Standby pools in MIGs](https://docs.cloud.google.com/compute/docs/instance-groups/accelerate-mig-scale-out-with-standby-pools.md.txt)** | Keep a standby pool of pre-configured suspended or stopped compute instances ready to speed up MIG scale-out | Compute Engine | MIG autoscaling, reservations | Manual resize or MIG scale-out demand that resumes or starts pooled compute instances | - Applications with slow operating system or runtime initialization - Sudden traffic spikes - Lower standby cost than active idle compute instances |
| **[Bulk compute instance creation](https://docs.cloud.google.com/compute/docs/instances/multiple/about-bulk-creation.md.txt)** | Create multiple compute instances in one API request with automatic zone and machine selection | Compute Engine | Reservations, Spot VMs, custom orchestrator | Single API or Google Cloud CLI bulk insert request | - One-time batch capacity bursts - Custom cluster deployments - Multi-instance hardware allocation |
| **[Sole-tenant node group autoscaling](https://docs.cloud.google.com/compute/docs/nodes/autoscaling-node-groups.md.txt)** | Autoscale dedicated physical servers for isolated workloads | Compute Engine and GKE | Reservations, MIG autoscaling, custom orchestrator, cluster autoscaler | Resource demand of guest compute instances | - Security and regulatory compliance - Per-core or per-socket license optimization - Hardware performance isolation |
| **[Custom orchestrator](https://docs.cloud.google.com/compute/docs/instances/instance-creation-overview.md.txt)** | Control compute instance creation, MIG target size, and deletion from your own scheduler, such as Ray, SkyPilot, or HTCondor | Compute Engine and Cluster Director | Reservations, MIG autoscaling, bulk mode and resize requests in MIGs, bulk compute instance creation, sole-tenant node group autoscaling, live cluster modification | Custom business metrics or external scheduler queues | - Proprietary scaling logic - Legacy on-premises scheduler migration - Fine-grained job lifecycle control |
| **[Slurm orchestration and dynamic nodes](https://docs.cloud.google.com/cluster-director/docs/orchestration.md.txt)** | Autoscale dynamic compute nodes in Slurm partitions for batch, AI, and HPC jobs | Cluster Director and Cluster Toolkit | Reservations, Flex-start VMs, Spot VMs, live cluster modification, live blueprint reconfiguration | Queued Slurm jobs and node idle timeouts | - HPC workloads - Large-scale AI/ML training - Batch simulation queues |
| **[Live cluster modification](https://docs.cloud.google.com/cluster-director/docs/manage/modify-cluster.md.txt)** | Add or resize partitions, nodesets, MIGs, or static and dynamic node limits on an active cluster | Cluster Director | Slurm orchestration and dynamic nodes, bulk mode and resize requests in MIGs, custom orchestrator, reservations | Cluster update request on an active cluster | - Baseline cluster expansion without job disruption - GPU, TPU, or CPU nodeset and MIG additions on active clusters |
| **[Live blueprint reconfiguration](https://docs.cloud.google.com/cluster-toolkit/docs/slurm/reconfigure-cluster.md.txt)** | Update and redeploy blueprints to add or resize Slurm partitions, static nodes, or GKE node pools on an active cluster | Cluster Toolkit | Slurm orchestration and dynamic nodes, cluster autoscaler, reservations | Blueprint update and redeployment on an active cluster | - Infrastructure-as-code cluster expansion without job disruption - Multi-reservation Slurm partitions |
| **[Custom ComputeClasses](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/about-custom-compute-classes.md.txt)** | Define prioritized hardware fallback rules, capacity consumption, and node migration | GKE in Standard and Autopilot modes | Cluster autoscaler, node pool auto-creation, Flex-start provisioning in GKE, reservations, capacity buffers | Pending Pods that match a `ComputeClass` selector or availability of higher-priority hardware | - High obtainability across GPU, TPU, or CPU series - Fallback from reservations to Flex-start or Spot VMs - Active migration back to preferred hardware |
| **[Cluster autoscaler](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler.md.txt)** | Add or remove nodes in existing node pools | GKE in Standard mode, and Cluster Toolkit GKE blueprints | Custom ComputeClasses, reservations, sole-tenant node group autoscaling, HPA, VPA, capacity buffers, scale to zero | Pending Pods or node underutilization | - Containerized workloads with variable traffic - Sudden spikes in demand |
| **[Node pool auto-creation](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/node-auto-provisioning.md.txt)** | Create and delete node pools in parallel to match Pod hardware requirements | GKE: managed in Autopilot mode; configurable in Standard mode | Custom ComputeClasses, reservations, Flex-start provisioning in GKE, HPA, VPA, capacity buffers | Pending Pods that require machine series, accelerators, or zones absent from existing node pools | - Diverse GPU, TPU, or CPU workload demands - Multi-tenant clusters that create multiple node pools in parallel |
| **[Flex-start provisioning in GKE](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/dws.md.txt)** | Obtain GPUs, TPUs, and H4D nodes as capacity opens, per node or all at once with queued provisioning | GKE in Standard and Autopilot modes | Custom ComputeClasses, node pool auto-creation, cluster autoscaler, Kueue job queues | Pending Pods or queued `ProvisioningRequest` objects when hardware capacity opens | - AI/ML model training and fine-tuning - Batch inference workloads - Simultaneous multi-node accelerator allocation |
| **[Capacity buffers](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/capacity-buffer.md.txt)** | Pre-provision spare active or suspended standby nodes for fast Pod placement and obtainability | GKE in Standard and Autopilot modes | Custom ComputeClasses, cluster autoscaler, node pool auto-creation, HPA, reservations | Consumption of buffered node capacity by scheduled Pods | - Latency-sensitive applications - AI inference and agent workloads - Sudden traffic spikes |
| **[Container-optimized compute platform](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview.md.txt#autopilot-compute-platform)** | Speed up node readiness and Pod placement with managed infrastructure and `Balanced` or `Scale-Out` ComputeClasses | GKE: Autopilot clusters, and Autopilot mode workloads in Standard clusters | Node pool auto-creation, HPA, VPA, capacity buffers, reservations | Pending Pods that require additional compute capacity | - Rapid cluster scale-out - Workloads with variable resource demand - Managed infrastructure operations |
| **[HPA](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/horizontalpodautoscaler.md.txt)** | Add or remove Pod replicas to match workload demand | GKE in Standard and Autopilot modes | Cluster autoscaler, node pool auto-creation, MPA, custom and external metrics, scale to zero | CPU usage, memory usage, or custom and external metrics | - Stateless applications - Web services and APIs - Variable request volume |
| **[VPA](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/verticalpodautoscaler.md.txt)** | Adjust container CPU and memory requests and limits based on usage | GKE in Standard and Autopilot modes | Cluster autoscaler, node pool auto-creation, MPA, in-place Pod resize, reservations | Historical container CPU and memory usage | - Stateful applications - Workloads with uncertain resource needs - Container resource optimization |
| **[MPA](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/multidimensional-pod-autoscaling.md.txt)** | Scale Pod replicas on CPU and container memory requests at the same time | GKE in Standard and Autopilot modes | Cluster autoscaler, node pool auto-creation, custom and external metrics | Simultaneous CPU demand shifts and container memory usage changes | - Stateless services with variable traffic and changing memory needs |
| **[Custom and external metrics](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/custom-and-external-metrics.md.txt)** | Scale Pod replicas based on application or external service signals | GKE in Standard and Autopilot modes | HPA, MPA, scale to zero, cluster autoscaler, node pool auto-creation | Application metrics or external queue and traffic metrics | - Queue-based workers, such as Pub/Sub consumers - Latency-bound web APIs - AI inference throughput targets |
| **[Scale to zero](https://docs.cloud.google.com/kubernetes-engine/docs/tutorials/scale-to-from-zero-hpa.md.txt)** | Scale idle Pods and node pools down to zero and wake them on demand or events | GKE in Standard and Autopilot modes | HPA, custom and external metrics, cluster autoscaler, node pool auto-creation | Absence of demand, or new demand in [HPA custom and external metrics](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/horizontal-pod-autoscaling.md.txt#scale-to-from-zero-hpa) or [KEDA triggers](https://docs.cloud.google.com/kubernetes-engine/docs/tutorials/scale-to-zero-using-keda.md.txt) | - Intermittent or bursty workloads - Event-driven batch jobs - Zero idle compute cost |
| **[Kueue job queues](https://docs.cloud.google.com/kubernetes-engine/docs/tutorials/kueue-intro.md.txt)** | Queue batch jobs, enforce team quotas, and coordinate simultaneous node allocation | GKE in Standard and Autopilot modes | Flex-start provisioning in GKE, Custom ComputeClasses, cluster autoscaler, node pool auto-creation | Admitted batch jobs when cluster quota and hardware capacity open | - Multi-team GPU and TPU quota management - Queued AI training and batch jobs |
| **[Cloud Storage FUSE with parallel streaming](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/cloud-storage-fuse-csi-driver.md.txt)** | Mount Cloud Storage buckets as file systems with parallel reads and caches | GKE in Standard and Autopilot modes | HPA, cluster autoscaler, node pool auto-creation, Pod snapshots | Pod startup and sequential file reads from mounted buckets | - High-throughput AI training and inference from Cloud Storage - Multi-node checkpoint access |
| **[CPU startup boost](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/boost-application-startup.md.txt)** | Grant temporary extra CPU to containers during initialization | GKE in Standard and Autopilot modes | HPA, VPA, cluster autoscaler, node pool auto-creation | Container startup and initialization phase | - Applications with high CPU needs at startup, such as Java Virtual Machine warmup - Latency-sensitive API scale-out |
| **[Image streaming](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/image-streaming.md.txt)** | Stream container image blocks on demand from Artifact Registry during Pod startup | GKE in Standard and Autopilot modes | HPA, cluster autoscaler, node pool auto-creation, capacity buffers | Container image pull from Artifact Registry when a Pod starts | - Large AI/ML container images - Fast microservice scale-out |
| **[Secondary boot disks](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/data-container-image-preloading.md.txt)** | Attach preloaded Persistent Disk volumes with container images or model data at node boot | GKE in Standard and Autopilot modes | Custom ComputeClasses, HPA, cluster autoscaler, node pool auto-creation, capacity buffers | Node boot in a node pool configured with secondary boot disk images | - Preloaded model weights or datasets on nodes - Base container image layer caching |
| **[In-place Pod resize](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/verticalpodautoscaler.md.txt#inplaceorrecreate_mode)** | Update active container CPU and memory allocations without Pod restarts | GKE in Standard and Autopilot modes | VPA, cluster autoscaler, node pool auto-creation | Updated container CPU or memory requests and limits | - Stateful or long-lived applications - Vertical scale without connection drops or state loss |
| **[Pod snapshots](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/pod-snapshots.md.txt)** | Restore Pod replicas from saved memory and file system checkpoints | GKE in Standard and Autopilot modes | HPA, cluster autoscaler, node pool auto-creation, capacity buffers | Pod replica creation from a saved snapshot | - AI inference servers with long model load times - Applications with slow runtime warmup - Rapid scale-out during demand spikes |
| **[Run:ai Model Streamer](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/run-ai-model-streamer.md.txt)** | Stream model tensors in parallel into GPU or TPU memory | GKE in Standard and Autopilot modes | HPA, cluster autoscaler, node pool auto-creation, Volume Populator | AI inference server startup and model load requests | - Large language model inference with vLLM - Fast GPU and TPU replica scale-out |
| **[Volume Populator](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/about-volume-populator.md.txt)** | Preload Hyperdisk ML volumes with data from Cloud Storage | GKE in Standard and Autopilot modes | HPA, cluster autoscaler, node pool auto-creation, Pod snapshots | `PersistentVolumeClaim` creation that references a `GCPDataSource` resource | - Preloaded AI/ML model weights - Shared read-only dataset access across multiple nodes |
| **[Autoscale policies](https://docs.cloud.google.com/vmware-engine/docs/howto-autoscale.md.txt)** | Add or remove ESXi hosts in a VMware private cloud cluster | VMware Engine | Cloud Load Balancing | Sustained CPU, memory, or vSAN storage usage | - Migrated VMware clusters with variable load - Storage-intensive or compute-intensive vSphere workloads |

## Best practices for scalability

To design infrastructure that scales without bottlenecks or capacity shortages,
apply the best practices in this section across compute capacity, network
design, and application architecture.

### Plan compute capacity and obtainability

To prepare your infrastructure for demand growth and improve hardware
obtainability, apply the following practices:

- **Estimate your workload needs** : before you configure autoscalers, measure
  your baseline usage and peak fluctuations. To inspect historical usage,
  forecast demand, and set accurate minimum and maximum compute instance
  limits, use
  [Capacity Planner](https://docs.cloud.google.com/capacity-planner/docs/overview.md.txt).

- **Design for resource obtainability** : when you request scarce hardware such
  as GPUs or TPUs,
  [optimize compute provisioning](https://docs.cloud.google.com/compute/docs/about-infrastructure-flexibility.md.txt)
  across the following three dimensions:

  - **Location flexibility** : distribute workloads across zones with
    [regional MIGs](https://docs.cloud.google.com/compute/docs/instance-groups/regional-migs.md.txt).

  - **Machine type flexibility** : allow multiple machine series through MIG
    instance flexibility or
    [GKE custom ComputeClasses](https://docs.cloud.google.com/kubernetes-engine/docs/best-practices/computeclasses.md.txt).

  - **Time flexibility**: use Flex-start provisioning when jobs can wait for
    capacity.

- **Use reservations for high capacity assurance**: for baseline traffic, peak
  scale events, or scarce GPUs and TPUs, reserve capacity in advance.
  Reservations provide a high level of assurance that resources are available
  in your target zone when your workloads scale out. If a single Slurm job
  exceeds one reservation, then span Cluster Toolkit partitions
  across multiple reservations.

- **Pre-warm spare capacity for latency-sensitive workloads**: if your
  application can't wait for new compute instances to provision during sudden
  traffic spikes, then maintain ready-to-use spare capacity. Configure
  GKE capacity buffers to pre-provision active or standby
  nodes, or use standby pools in Compute Engine MIGs to resume
  suspended compute instances without full operating system boot delays.

### Design networks for scale

To move data between scaled resources without delays or address exhaustion,
apply the following network practices:

- **Allocate sufficient IP addresses**: verify that your subnets have enough
  free IP addresses for the maximum instance or node count in your autoscaler
  limits. In GKE, also check secondary IP ranges for Pods and
  Services. If a subnet runs out of IP addresses, then autoscalers fail to
  create new compute instances or Pods.

- **Choose high-throughput network interfaces** : for workloads that transfer
  large datasets or model weights across nodes, select high-bandwidth
  [network interface controllers](https://docs.cloud.google.com/compute/docs/networking/network-overview.md.txt#nics)
  such as Google Virtual Network Interface Card or remote direct memory
  access. These interfaces reduce CPU overhead and increase bandwidth between
  scaled compute instances.

- **Optimize instance placement** : for multi-node AI training or HPC jobs,
  group compute instances on physical servers close together with
  [compact placement policies](https://docs.cloud.google.com/compute/docs/instances/use-compact-placement-policies.md.txt)
  or topology-aware placement in Cluster Director and
  Cluster Toolkit. Close physical placement reduces network
  hops and latency across scaled nodes. For fault-tolerant web services,
  spread compute instances across zones instead so that a single zone outage
  doesn't disrupt traffic.

### Design applications for scale

To let your applications scale out and scale in without user disruption, apply
the following architecture practices:

- **Keep application tiers stateless**: store user session data, checkpoints,
  and shared files in external storage or databases rather than on local
  instance disks. This design lets autoscalers add or remove compute instances
  and Pods at any time without data loss.

- **Decouple components with queues and microservices**: split monolithic
  workloads into independent services connected by load balancers or message
  queues such as Pub/Sub. This architecture lets you scale each
  service based on its own demand---including down to zero with
  event-driven autoscalers---and absorb traffic bursts without overload on
  downstream backends.

- **Shorten container startup and model load times**: reduce the delay
  between resource creation and traffic readiness so that scaled capacity
  serves requests right away. In GKE, stream or preload
  container images and use CPU startup boost or Pod snapshots to speed up
  runtime initialization. For AI workloads, stream model weights from
  Cloud Storage FUSE, Run:ai Model Streamer, or pre-populated volumes.