This document compares the available consumption options and helps you choose the option that best fits your workload and cost needs.
When you create a cluster in Cluster Director, you must select the consumption option that you want to use to obtain resources (vCPUs, memory, GPUs, TPUs, and Local SSD disks) for creating Compute Engine instances. Each consumption option defines the availability, lifespan, and cost of your requested resources.
Compare consumption options
The following table compares the supported machine types, lifespans, capacity assurance levels, pricing models, and reservation requirements for each consumption option:
| Consumption option | Supported machines | Lifespan | Capacity assurance | Quota | Pricing | Billing model | Resource allocation | Provisioning model |
|---|---|---|---|---|---|---|---|---|
| Future reservations for capacity blocks | A4X Max, A4X, A4, A3 Ultra, A3 Mega, H4D, TPU7x, TPU v6e, TPU v5p | Unlimited | Very high | Google increases quota on your behalf | Discounted up to 53% (VM instance pricing, TPU pricing) | Charged for the reservation period (reservation billing) | Dense for GPU, H4D, TPU7x, and TPU v6e VMs; best-effort for TPU v5p VMs | Reservation-bound |
| Future reservations in calendar mode | A4, A3 Ultra, A3 Mega, H4D, TPU7x, TPU v6e, TPU v5p | 90 days maximum | Very high | No quota is charged for GPU or TPU VMs; standard CPU quota is charged for H4D VMs | Discounted up to 53% (Dynamic Workload Scheduler pricing) | Charged for the reservation period (reservation billing) | Dense for GPU and H4D VMs, best-effort for TPU VMs | Reservation-bound |
| Future reservations | A3 Mega, G4, H4D, N2 | Unlimited | Very high | Standard quota is charged | Standard pricing (discounted with commitments) | Charged for the reservation period (reservation billing) | Best-effort | Standard |
| On-demand reservations | A3 Mega, G4, H4D, N2 | Unlimited | Very high | Standard quota is charged | Standard pricing (discounted with commitments) | Charged for the reservation period (reservation billing) | Best-effort (compact placement policies optional) | Standard |
| Flex-start | A4, A3 Ultra, A3 Mega, G4, H4D, TPU7x, TPU v6e, TPU v5p | 7 days maximum | Best-effort | Preemptible quota is charged | Discounted up to 53% (Dynamic Workload Scheduler pricing) | Pay-as-you-go (PAYG) | Best-effort (workload policies applied for TPU VMs) | Flex-start |
| Spot | A4, A3 Ultra, A3 Mega, G4, H4D, N2, TPU7x, TPU v6e, TPU v5p | Unlimited (but subject to preemption) | Best-effort | Preemptible quota is charged | Discounted up to 91% (Spot pricing) | Pay-as-you-go (PAYG) | Best-effort (workload policies applied for TPU VMs; compact placement policies applied for GPU, H4D, and N2 VMs in Slurm clusters) | Spot |
| On-demand | A3 Mega, C4A, G4, H4D, N2, TPU7x, TPU v6e, TPU v5p | Unlimited | Best-effort | Standard quota is charged | Standard pricing | Pay-as-you-go (PAYG) | Best-effort (workload policies applied for TPU VMs) | Standard |
If you want AI-powered assistance to compare consumption options, then run the following prompt in Compute Advisor:
This prompt evaluates your project's quotas and reservations, compares capacity assurance, lifespan, and discounts across consumption options, and outputs a Markdown comparison table with recommended partition configurations.
Choose a consumption option
To choose the consumption option that best fits the type of partition for your cluster, answer the following questions:
Do you need reserved capacity for a high assurance of obtaining compute instances?
Yes: go to question 2.
No: skip to question 5.
Does your GPU, H4D, or TPU workload require densely allocated resources?
Yes: go to question 3.
No: skip to question 4.
Do you need the reserved capacity for more than 90 days?
Do you want to reserve capacity for a future date instead of immediately?
Yes: see Use future reservations.
No: see Use on-demand reservations.
Is your workload fault-tolerant and able to handle VM preemption?
Yes: see Use Spot.
No: go to question 6.
Does your GPU, H4D, or TPU workload run for 7 days or less?
Yes: see Use Flex-start.
No: see Use On-demand.
Use future reservations for capacity blocks
To run long-running, large-scale distributed workloads that require densely allocated resources, you can reserve vCPUs, memory, GPUs, TPUs, Local SSD disks, and a Google Cloud Hyperdisk pool for a future period. If Google approves your request, then you have exclusive access to your reserved resources for that period of time, and you can use the resources to create clusters. At the end of the reservation period, Compute Engine does the following:
Compute Engine deletes the reservation and any associated empty Hyperdisk pool.
Compute Engine deletes any compute instances in your cluster that use the reservation.
Ideal workloads for future reservations for capacity blocks
Future reservations for capacity blocks are ideal for the following workloads:
A4X Max, A4X, and A4: pre-training foundation and mixture-of-experts (MoE) models, multi-host foundation model inference, and large-scale scientific simulations.
A3 Ultra and A3 Mega: long-running large model training, continuous fine-tuning pipelines, multi-host inference serving, and HPC simulations.
H4D: large-scale HPC simulations and distributed training with message passing interface (MPI).
TPU7x and TPU v5p: pre-training massive foundation models from scratch, continuous reinforcement learning loops, and recommendation models with large embedding tables.
TPU v6e: continuous training or fine-tuning pipelines and high-throughput production inference serving.
Characteristics of future reservations for capacity blocks
Future reservations for capacity blocks have the following characteristics:
You can reserve A4X Max, A4X, A4, A3 Ultra, A3 Mega, H4D, TPU7x, TPU v6e, and TPU v5p machine types, as well as Hyperdisk pools. For GPU, H4D, TPU7x, and TPU v6e VMs, Google allocates compute nodes in close physical proximity to minimize network latency. Google also co-locates Hyperdisk pools with those compute nodes to optimize storage performance. When you create TPU VMs from your reservation, Cluster Director applies a workload policy to provision all VMs within a slice together on a 2D or 3D interconnect topology.
You can reserve as many compute instances as you want for as long as you want. For any compute instances that you want to reserve for a year or longer, you must purchase and attach a resource-based commitment to your reserved resources. Then, you can use the reserved resources to create and run compute instances until the end of the reservation period.
You use the reservation-bound provisioning model to create compute instances, which has the following benefits:
You have a higher chance to obtain GPU, H4D, or TPU VMs.
Depending on your reservation duration and any attached commitments, you get a discount up to 53% for vCPUs, memory, GPUs, TPUs, and any attached Local SSD disks. If you reserve resources for 365 days or longer, then you also get discounts for Hyperdisk pools.
How to use future reservations for capacity blocks
To use future reservations for capacity blocks to create clusters, you must complete the following steps:
Request to reserve capacity. Contact your account team and specify the compute instances and Hyperdisk pools to reserve. Based on availability, Google creates a draft reservation request for you. If the draft matches your requirements, then accept and submit the request. If you reserve compute and storage capacity together, then you must accept both requests as a bundle to ensure that Google delivers both resources at the same time. Google Cloud approves your reservation request within a few minutes.
For instructions, see Reserve capacity through your account team.
Consume reserved resources. At the start of your reservation period, Google delivers your compute reservation and Hyperdisk pools. You can use the reserved resources to create compute instances and attach storage disks in one or more of your cluster partitions. For reservations of A4X, H4D, TPU7x, and TPU v6e instances, you can also select specific blocks or sub-blocks to control the placement of your VMs.
For instructions, see one of the following:
Use future reservations in calendar mode
To run short-running distributed workloads that require densely allocated resources, you can request to reserve compute resources for up to 90 days. If Google approves your request, then you have exclusive access to your reserved resources for that time, and you can use the resources to create clusters. At the end of the reservation period, Compute Engine does the following:
Compute Engine deletes the reservation.
Compute Engine deletes any VM instances that use the reservation.
Ideal workloads for future reservations in calendar mode
Future reservations in calendar mode are ideal for the following workloads:
A4, A3 Ultra, and A3 Mega: time-boxed model pre-training, fine-tuning, scientific simulations, and batch inference or model evaluation.
H4D: HPC simulations and distributed training with MPI.
TPU7x and TPU v5p: time-boxed model training runs, fine-tuning, and reinforcement learning.
TPU v6e: cost-effective fine-tuning, distillation, and high-throughput batch inference.
Characteristics of future reservations in calendar mode
Future reservations in calendar mode have the following characteristics:
You can reserve A4, A3 Ultra, A3 Mega, H4D, TPU7x, TPU v6e, or TPU v5p machine types. Google densely allocates GPU and H4D VMs to minimize network latency, and allocates TPU VMs as close to each other as possible on a best-effort basis. When you create TPU VMs from your reservation, Cluster Director applies a workload policy to provision all VMs within a slice together on a 2D or 3D interconnect topology.
You can view the future availability of resources, and then reserve GPU, H4D, or TPU capacity for up to 90 days. At your chosen delivery date, you can use the reserved resources to create VMs until the end of the reservation period.
You use the reservation-bound provisioning model to create compute instances, which has the following benefits:
You have a higher chance to obtain GPU, H4D, or TPU VMs.
You get a discount up to 53% for vCPUs, memory, GPUs, TPUs, and any attached Local SSD disks.
How to use future reservations in calendar mode
To use future reservations in calendar mode to create clusters, you must complete the following steps:
View resource availability. You can view the future availability of the resources that you want to reserve. When you create a reservation request, you can specify the number, type, and reservation duration for the resources that you confirmed as available. This action increases the chances that Google Cloud approves your request.
For instructions, see View the future availability of resources.
Reserve capacity. You create a reservation request for a future date and time. Google Cloud approves the reservation request within one minute. If Google Cloud approves the request, then Compute Engine reserves the capacity for you. At your chosen delivery date, you can use the reserved resources to create clusters.
For instructions, see Create a request for GPU VMs, H4D VMs, or TPUs.
Consume reserved resources. At the start of your chosen reservation period, you can use the reservation to create VMs in one or more of your cluster partitions.
For instructions, see one of the following:
Use future reservations
To obtain high capacity assurance for A3 Mega, G4, H4D, or N2 VMs starting on a specific future date and time, you can create and submit a future reservation request. If Google Cloud approves your request, then Compute Engine automatically creates on-demand reservations with your requested capacity on your chosen delivery date and time. You have exclusive access to your reserved resources, and you can use the auto-created reservations to create clusters. After the reservation period ends, you can modify or delete the auto-created reservations at any time. Any VMs that consume the reservations continue to run until you stop or delete the VMs.
Ideal workloads for future reservations
Future reservations are ideal for the following workloads:
A3 Mega: scheduled inference serving for peak-traffic events, single-host or loosely coupled fine-tuning, and batch jobs where you want to keep the auto-created reservation after the reservation period ends.
G4: scheduled single-host inference, model fine-tuning, graphics-intensive applications, and video transcoding.
H4D: scheduled HPC simulations and distributed MPI workloads where you want to keep the auto-created reservation after the reservation period ends.
N2: planned CPU-bound HPC jobs, data preprocessing pipelines, and cluster login nodes.
Characteristics of future reservations
Future reservations have the following characteristics:
You can reserve A3 Mega, G4, H4D, and N2 machine types. Compute Engine closely allocates your reserved resources on a best-effort basis. For G4 VMs, your reservation must include 4 Local SSD disks per GPU that use the NVMe disk interface.
You can request capacity for at least 24 hours and starting up to one year in the future. Cluster Director supports only specifically targeted reservations.
You must have sufficient standard quota for your requested resources both when you submit your future reservation request for review and when your request reaches its start time.
You use the standard provisioning model to create VMs. Standard pricing applies for the entire reservation period, and you can attach committed use discounts (CUDs).
How to use future reservations
To use future reservations to create clusters, complete the following steps:
Create and submit a future reservation request. Create a future reservation request for A3 Mega, G4, H4D, or N2 VMs, and set the reservation affinity to specifically targeted (
SPECIFIC_RESERVATION). If you request G4 VMs, then include 4 Local SSD disks per GPU that use the NVMe disk interface. Then, submit the request to Google Cloud for review.For instructions, see one of the following:
Consume reserved resources. At the start time of your reservation period, Compute Engine provisions the auto-created reservations. You can then target the auto-created reservations to create VMs in one or more of your cluster partitions.
For instructions, see one of the following:
Use on-demand reservations
To immediately reserve capacity for A3 Mega, G4, H4D, or N2 VMs and hold it for as long as you need, you can create an on-demand reservation. If the requested capacity is available in your specified zone, then Compute Engine creates the reservation within a few minutes. You have exclusive access to your reserved resources until you delete the reservation.
Ideal workloads for on-demand reservations
On-demand reservations are ideal for the following workloads:
A3 Mega: production inference, model serving, and fine-tuning that require assured capacity or frequently stop and restart VMs.
G4: production single-host inference, model fine-tuning, graphics-intensive applications, and video transcoding that require assured capacity.
H4D: tightly coupled HPC simulations and MPI workloads that require assured capacity and flexible reservation duration.
N2: critical HPC workloads, login nodes, and CPU-bound tasks that require a high assurance for capacity.
Characteristics of on-demand reservations
On-demand reservations have the following characteristics:
You can reserve up to 1,000 A3 Mega, G4, H4D, or N2 VMs per reservation. Cluster Director supports only specifically targeted reservations. For G4 VMs, your reservation must include 4 Local SSD disks per GPU that use the NVMe disk interface.
Compute Engine allocates your reserved resources on a best-effort basis. To minimize network latency, you can optionally apply a compact placement policy to a single-project, specifically targeted reservation that isn't attached to a commitment.
You must have sufficient standard quota in your project before you create the reservation. Compute Engine charges your quota immediately when it creates the reservation.
You use the standard provisioning model to create VMs. You can modify or delete an on-demand reservation at any time, unless it's attached to a commitment. Standard pricing applies for as long as the reservation exists, and you can attach CUDs for savings.
How to use on-demand reservations
To use on-demand reservations to create clusters, complete the following steps:
Create an on-demand reservation. Create a specifically targeted reservation for A3 Mega, G4, H4D, or N2 VMs. If you reserve G4 VMs, then include 4 Local SSD disks per GPU that use the NVMe disk interface. To create VMs in close physical proximity and minimize network latency, you can optionally specify a compact placement policy when you create the reservation.
For instructions, see one of the following:
Consume reserved resources. Target your reservation to create VMs in one or more of your cluster partitions. When you create VMs in your cluster, Cluster Director automatically applies a compact placement policy to the VMs if the reservation specifies a compact placement policy.
For instructions, see one of the following:
Use Flex-start
To run short-duration workloads that require densely allocated resources, you can request compute resources for up to seven days by using Flex-start. Whenever resources are available, Compute Engine creates your requested number of VMs. The Flex-start VMs run until you delete them, or until Compute Engine deletes the VMs at the end of their run duration.
Ideal workloads for Flex-start
Flex-start is ideal for workloads that can start at any time, such as the following:
A4, A3 Ultra, and A3 Mega: short-duration model pre-training, fine-tuning, model evaluation, HPC simulations, and batch inference.
G4: short-duration model fine-tuning, batch inference, graphics-intensive applications, and video transcoding.
H4D: short-duration HPC simulations and distributed MPI workloads.
TPU7x, TPU v6e, and TPU v5p: short-duration model training, fine-tuning, distillation, and batch inference.
Characteristics of Flex-start
Flex-start has the following characteristics:
You can request A4, A3 Ultra, A3 Mega, G4, H4D, TPU7x, TPU v6e, and TPU v5p VMs. For TPU VMs, Cluster Director applies a workload policy to provision all VMs within a slice together on a 2D or 3D interconnect topology. For GPU and H4D VMs, Compute Engine allocates VMs as close to each other as possible on a best-effort basis.
You use the flex-start provisioning model to create Flex-start VMs, which has the following benefits:
You have a higher chance to obtain GPU, H4D, or TPU VMs.
You get a discount up to 53% for vCPUs, memory, GPUs, TPUs, and any attached Local SSD disks.
You can request resources for up to seven days. At the end of the maximum run duration, Compute Engine deletes the VMs. In Slurm clusters, Cluster Director automatically requests new resources with the same run duration. In managed compute clusters, Compute Engine doesn't replace deleted Flex-start VMs.
How to use Flex-start
To use Flex-start to create VMs in one or more of your cluster partitions, use one of the following methods:
Use Spot
To run fault-tolerant workloads, you can obtain compute resources immediately based on availability. You get resources at the lowest price possible. However, Compute Engine can preempt Spot VMs at any time to reclaim capacity.
Ideal workloads for Spot
Spot is ideal for workloads where interruptions are acceptable, such as the following:
A4, A3 Ultra, and A3 Mega: fault-tolerant distributed training with frequent checkpointing, batch inference, and interruptible HPC simulations.
G4: fault-tolerant batch inference, model fine-tuning, graphics-intensive rendering, and video transcoding.
TPU7x, TPU v6e, and TPU v5p: fault-tolerant model training, batch inference, and reinforcement learning with checkpointing.
H4D: fault-tolerant HPC simulations and distributed MPI jobs with checkpointing.
N2: batch processing, continuous integration and continuous deployment (CI/CD), data preprocessing, and fault-tolerant HPC jobs.
Characteristics of Spot
Spot has the following characteristics:
You can create A4, A3 Ultra, A3 Mega, G4, H4D, N2, TPU7x, TPU v6e, and TPU v5p VMs. For TPU VMs, Cluster Director applies a workload policy to provision all VMs within a slice together on a 2D or 3D interconnect topology. In Slurm clusters, Cluster Director automatically applies a compact placement policy to place GPU, H4D, and N2 Spot VMs as close to each other as possible. Slice availability and dense allocation across nodes depend on resource availability.
You can immediately create clusters. Unless Compute Engine preempts the VMs to reclaim capacity, GPU, H4D, and N2 VMs run until you stop or delete them, and TPU VMs run until you delete them. If you use GPU, H4D, or N2 Spot VMs in a managed compute cluster, then you can only configure Compute Engine to stop (
TERMINATE) them on preemption.You use the spot provisioning model to create Spot VMs, which has the following benefits:
You have a higher chance to obtain GPU, H4D, or TPU VMs.
You get a discount up to 91% for vCPUs, memory, and any attached GPUs, TPUs, and Local SSD disks.
How to use Spot
To use Spot to create VMs in one or more of your cluster partitions, complete the following steps:
View resource availability. You can view the availability of A4, A3 Ultra, A3 Mega, G4, H4D, and N2 machine types in the region or zone where you want to create Spot VMs. This action helps reduce the time that Cluster Director takes to allocate resources and create Spot VMs.
For instructions, see View resource availability for Spot VMs.
View preemption rate and pricing. You can view the historical and current preemption rate and pricing across A4, A3 Ultra, A3 Mega, G4, H4D, and N2 machine types and zones. This information helps you choose the machine type and location that best fit your workload and budget.
For instructions, see View the preemption rate and pricing for Spot VMs.
Create your cluster. To create your cluster with one or more partitions that use Spot VMs, select one of the following methods:
Use On-demand
To run workloads on A3 Mega, C4A, G4, H4D, N2, TPU7x, TPU v6e, and TPU v5p VMs without a reservation, you can request on-demand resources based on real-time availability at standard pricing.
Ideal workloads for On-demand
On-demand is ideal for the following workloads:
A3 Mega: small-scale prototyping, testing, and single-host inference that don't require assured capacity and can tolerate provisioning delays.
G4: lightweight development, testing, single-host inference, model fine-tuning, graphics-intensive applications, and video transcoding.
TPU7x, TPU v6e, and TPU v5p: small-slice testing, prototyping, and inference or fine-tuning jobs that don't require assured capacity and can tolerate provisioning delays.
C4A: Arm-based login nodes for clusters with A4X Max or A4X compute nodes.
H4D: tightly coupled HPC simulations and CPU-bound computational tasks.
N2: login nodes, CPU-bound computational tasks, and general-purpose HPC.
Characteristics of On-demand
On-demand has the following characteristics:
You can create A3 Mega, C4A, G4, H4D, N2, TPU7x, TPU v6e, and TPU v5p VMs. For TPU VMs, Cluster Director applies a workload policy to provision all VMs within a slice together on a 2D or 3D interconnect topology. For A3 Mega, C4A, G4, H4D, and N2 VMs, dense allocation depends on resource availability. You can use C4A VMs only for login nodes in Slurm clusters.
You can request VMs without advance reservations or long-term commitments, and resource allocation depends on the real-time availability of hardware. GPU and CPU VMs run until you stop or delete them, and TPU VMs run until you delete them.
You use the standard provisioning model, which is the default model. You pay standard VM instance pricing or TPU pricing only while the VMs run.
How to use On-demand
To use On-demand to create VMs in one or more of your cluster partitions, use one of the following methods: