When you create a cluster in Cluster Director, you must select the *consumption option* that you want to use to obtain resources (vCPUs, memory, GPUs, TPUs, and Local SSD disks) for creating Compute Engine instances. Each consumption option defines the availability, lifespan, and cost of your requested resources. This document compares the available consumption options, and it helps you choose the option that best fits your workload and cost needs. ## Compare consumption options > [!NOTE] > **Note:** Support for TPU VMs in Cluster Director is in [Preview](https://cloud.google.com/products#product-launch-stages). The following table compares the supported machine types, lifespans, capacity assurance levels, pricing models, and reservation requirements for each consumption option: | Consumption option | Future reservations for capacity blocks | Future reservations in calendar mode (up to 90 days) | Flex-start | Spot | On-demand | |---|---|---|---|---|---| | **Supported machines** | A4X Max, A4X, A4, A3 Ultra, A3 Mega, TPU7x^\*^, TPU v6e, TPU v5p | A4, A3 Ultra, A3 Mega, TPU7x^\*^, TPU v6e, TPU v5p | A4, A3 Ultra, A3 Mega, TPU7x^\*^, TPU v6e, TPU v5p | A4, A3 Ultra, A3 Mega, TPU7x^\*^, TPU v6e, TPU v5p, N2 | C4A, N2, TPU7x, TPU v6e, TPU v5p | | **Lifespan** | Unlimited | 90 days maximum | 7 days maximum | Unlimited (but subject to [preemption](https://docs.cloud.google.com/compute/docs/instances/spot.md.txt#preemption)) | Unlimited | | **Capacity assurance** | Very high | Very high | Best-effort | Best-effort | Best-effort | | **Quota** | Google increases quota on your behalf | No quota is charged | [Preemptible quota](https://docs.cloud.google.com/compute/resource-usage.md.txt#preemptible-quotas) is charged | [Preemptible quota](https://docs.cloud.google.com/compute/resource-usage.md.txt#preemptible-quotas) is charged | Standard quota is charged | | **Pricing** | Discounted up to 53% ([accelerator pricing](https://docs.cloud.google.com/compute/vm-instance-pricing#accelerator-optimized), [TPU pricing](https://cloud.google.com/tpu/pricing)) | Discounted up to 53% ([DWS pricing](https://cloud.google.com/products/dws/pricing)) | Discounted up to 53% ([DWS pricing](https://cloud.google.com/products/dws/pricing)) | Deeply discounted up to 91% ([Spot pricing](https://cloud.google.com/spot-vms/pricing)) | Standard pricing ([VM instance pricing](https://docs.cloud.google.com/compute/vm-instance-pricing), [TPU pricing](https://cloud.google.com/tpu/pricing)) | | **Billing model** | Charged for the reservation period ([reservation billing](https://docs.cloud.google.com/compute/docs/instances/reservations-overview.md.txt#reservation-billing)) | Charged for the reservation period ([reservation billing](https://docs.cloud.google.com/compute/docs/instances/reservations-overview.md.txt#reservation-billing)) | Pay-as-you-go (PAYG) | Pay-as-you-go (PAYG) | Pay-as-you-go (PAYG) | | **Resource allocation** | Dense (best-effort for TPU v5p) | Dense for GPU virtual machines (VMs), best-effort for TPU VMs | Best-effort | Best-effort | Best-effort | | [**Provisioning model**](https://docs.cloud.google.com/compute/docs/instances/provisioning-models.md.txt) | Reservation-bound | Reservation-bound | Flex-start | Spot | Standard | ^\*^ To obtain TPU7x VMs with any consumption option except On-demand, you must contact your account team or the sales team to request access. If you want AI-powered assistance to compare consumption options, then run the following prompt in Compute Advisor: ``` Act as a cloud architect. Help me choose the best consumption option and provisioning model for each partition in my Cluster Director cluster based on my project quotas, existing reservations, and available regions. Walk me through the following steps: 1. Compare the five consumption options (future reservations for capacity blocks, future reservations in calendar mode, Flex-start, Spot, and On-demand) across supported machine series, lifespan, capacity assurance, quota type (standard or preemptible), pricing discounts, billing model, and resource allocation. 2. Evaluate regional and zonal availability for GPU machine series (A4X Max, A4X, A4, A3 Ultra, and A3 Mega), TPU machine series, and CPU machine series (N2 for compute or controller nodes, and C4A or N2 for login nodes). 3. Recommend a consumption option and provisioning model (reservation-bound, flex-start, spot, or standard) for each cluster partition based on whether my workload needs high assurance, capacity for more than 90 days, reserved capacity for up to 90 days, a run duration of up to 7 days, or fault-tolerant Spot capacity. Format the recommendations as a Markdown comparison table for my Slurm partitions and nodesets, followed by the steps to reserve capacity and create the cluster. ``` ## Choose a consumption option Use the following flowchart to choose the consumption option that best fits the type of partition for your cluster: [![Decision flowchart that maps capacity assurance, duration, reservation, and fault-tolerance requirements to Cluster Director consumption options: future reservations for capacity blocks, future reservations in calendar mode, Spot, Flex-start, and On-demand.](https://docs.cloud.google.com/static/cluster-director/images/consumption-options.png)](https://docs.cloud.google.com/static/cluster-director/images/consumption-options.png) The questions in the preceding diagram are the following: 1. Do you want high assurance for GPU or TPU instances? - **Yes**: go to question 2. - **No**: skip to question 4. 2. Do you need capacity for more than 90 days? - **Yes** : see [Use future reservations for capacity blocks](https://docs.cloud.google.com/cluster-director/docs/choose-consumption-option#future-reservations). - **No**: go to question 3. 3. Do you want reserved capacity? - **Yes** : see [Use future reservations in calendar mode](https://docs.cloud.google.com/cluster-director/docs/choose-consumption-option.md.txt#calendar-mode). - **No**: go to question 4. 4. Is your workload fault-tolerant? - **Yes** : see [Use Spot](https://docs.cloud.google.com/cluster-director/docs/choose-consumption-option.md.txt#spot). - **No**: go to question 5. 5. Is your workload running for 7 days or less? - **Yes** : see [Use Flex-start](https://docs.cloud.google.com/cluster-director/docs/choose-consumption-option.md.txt#flex-start). - **No** : see [Use On-demand](https://docs.cloud.google.com/cluster-director/docs/choose-consumption-option.md.txt#on-demand). ### Use future reservations for capacity blocks To run long-running, large-scale distributed workloads that require densely allocated resources, you can reserve vCPUs, memory, GPUs, TPUs, Local SSD disks, and a [Google Cloud Hyperdisk pool](https://docs.cloud.google.com/compute/docs/disks/pools.md.txt) for a future period. If Google approves your request, then you have exclusive access to your reserved resources for that period of time, and you can use the resources to create clusters. At the end of the reservation period, Compute Engine does the following: - Compute Engine deletes the reservation and any associated empty Hyperdisk pool. - Compute Engine deletes any compute instances in your cluster that use the reservation. > [!CAUTION] > **Caution:** When the reservation that you used to create a compute instance reaches its end time, Cluster Director deletes the instance. To prevent data loss, back up your data by using one of the [data protection options for disks and instances](https://docs.cloud.google.com/compute/docs/disks/data-protection.md.txt). #### Ideal workloads for future reservations for capacity blocks Future reservations for capacity blocks are ideal for the following workloads: - Pre-training foundation models - Inference for foundation models across multiple hosts #### Characteristics of future reservations for capacity blocks Future reservations for capacity blocks have the following characteristics: - You can reserve A4X Max, A4X, A4, A3 Ultra, A3 Mega, TPU7x, TPU v6e, and TPU v5p machine types, as well as Hyperdisk pools. For GPU, TPU7x, and TPU v6e VMs, Google allocates compute nodes in close physical proximity to minimize network latency, and co-locates Hyperdisk pools with those compute nodes to optimize storage performance. > [!NOTE] > **Note:** To reserve TPU7x VMs with future reservations for capacity blocks, you must contact your account team or the sales team to request access. - You can reserve as many compute instances as you want for as long as you want. For any compute instances that you want to reserve for a year or longer, you need to purchase and attach a [resource-based commitment](https://docs.cloud.google.com/compute/docs/instances/reservations-with-commitments) to your reserved resources. Then, you can use the reserved resources to create and run compute instances until the end of the reservation period. - You use the reservation-bound provisioning model to create compute instances, which has the following benefits: - You have a higher chance to obtain GPUs or TPUs. - In addition to the commitment that is attached to your compute instances, you get a discount up to 53% for vCPUs, memory, GPUs, TPUs, and any attached Local SSD disks. If you reserve resources for 365 days or longer, then you also get discounts for Hyperdisk pools. #### How to use future reservations for capacity blocks To use future reservations for capacity blocks to create clusters, you must complete the following steps: 1. **Request to reserve capacity**. Contact your account team and specify the compute instances and Hyperdisk pools to reserve. Based on availability, Google creates a draft reservation request for you. If the draft matches your requirements, then accept and submit the request. If you reserve compute and storage capacity together, then you must accept both requests as a bundle to ensure that Google delivers both resources at the same time. Google Cloud approves your reservation request within a few minutes. For instructions, see [Reserve capacity through your account team](https://docs.cloud.google.com/cluster-director/docs/reserve-capacity). 2. **Consume reserved resources** . At the start of your reservation period, Google delivers your compute reservation and Hyperdisk pools. You can use the reserved resources to create compute instances and attach storage disks in one or more of your cluster partitions. For A4X, TPU7x, and TPU v6e reservations (in [all-capacity mode](https://docs.cloud.google.com/tpu/docs/all-capacity-overview) for TPUs), you can also select specific [blocks](https://docs.cloud.google.com/ai-hypercomputer/docs/terminology.md.txt#block) or [sub-blocks](https://docs.cloud.google.com/ai-hypercomputer/docs/terminology.md.txt#sub-block) to control the placement of your VMs. For instructions, see one of the following: - [Create an AI-optimized cluster based on a template](https://docs.cloud.google.com/cluster-director/docs/create/cluster-from-template.md.txt) - [Create a custom cluster](https://docs.cloud.google.com/cluster-director/docs/create/custom-cluster.md.txt) ### Use future reservations in calendar mode To run short-running distributed workloads that require densely allocated resources, you can request to reserve compute resources for up to 90 days. If Google approves your request, then you have exclusive access to your reserved resources for that time, and you can use the resources to create clusters. At the end of the reservation period, Compute Engine does the following: - Compute Engine deletes the reservation. - Compute Engine deletes any VM instances that use the reservation. > [!CAUTION] > **Caution:** After the reservation that you used to create a VM reaches its end time, Cluster Director deletes the VM regardless of its [termination > action](https://docs.cloud.google.com/compute/docs/instances/limit-vm-runtime#limit-runtime-new-vm). To prevent data loss, back up your data by using one of the data protection options. For more information, see [Data protection options for disks and > instances](https://docs.cloud.google.com/compute/docs/disks/data-protection). #### Ideal workloads for future reservations in calendar mode Future reservations in calendar mode are ideal for the following workloads: - Model pre-training - Model fine-tuning - Simulations - Inference #### Characteristics of future reservations in calendar mode Future reservations in calendar mode have the following characteristics: - You can reserve A4, A3 Ultra, A3 Mega, TPU7x^\*^, TPU v6e, or TPU v5p machine types. Google densely allocates GPU VMs to minimize network latency, and allocates TPU VMs as close to each other as possible on a best-effort basis. > [!NOTE] > **Note:** To reserve TPU7x VMs with future reservations in calendar mode, you must contact your account team or the sales team to request access. - You can view the future availability of resources, and then reserve GPU or TPU capacity for up to 90 days. At your chosen delivery date, you can use the reserved resources to create VMs until the end of the reservation period. For supported request sizes and locations, see [Limitations for future reservation requests in calendar mode](https://docs.cloud.google.com/compute/docs/instances/future-reservations-calendar-mode-overview#limitations) in the Compute Engine documentation. - You use the reservation-bound provisioning model to create compute instances, which has the following benefits: - You have a higher chance to obtain GPUs or TPUs. - You get a discount up to 53% for vCPUs, memory, GPUs, TPUs, and any attached Local SSD disks. #### How to use future reservations in calendar mode To use future reservations in calendar mode to create clusters, you must complete the following steps: 1. **View resource availability**. You can view the future availability of the resources that you want to reserve. When you create a reservation request, you can specify the number, type, and reservation duration for the resources that you confirmed as available. This action increases the chances that Google Cloud approves your request. For instructions, see [View the future availability of resources](https://docs.cloud.google.com/compute/docs/instances/create-future-reservations-calendar-mode#view-availability). 2. **Reserve capacity**. You create a reservation request for a future date and time. Google Cloud approves the reservation request within two minutes. If Google Cloud approves the request, then Compute Engine reserves the capacity for you. At your chosen delivery date, you can use the reserved resources to create clusters. For instructions, see [Create a request for GPU VMs, H4D VMs, or TPUs](https://docs.cloud.google.com/compute/docs/instances/create-future-reservations-calendar-mode#request). 3. **Consume reserved resources**. At the start of your chosen reservation period, you can use the reservation to create VMs in one or more of your cluster partitions. For instructions, see one of the following: - [Create an AI-optimized cluster based on a template](https://docs.cloud.google.com/cluster-director/docs/create/cluster-from-template.md.txt) - [Create a custom cluster](https://docs.cloud.google.com/cluster-director/docs/create/custom-cluster.md.txt) ### Use Flex-start To run short-duration workloads that require densely allocated resources, you can request compute resources for up to seven days by using Flex-start. Whenever resources are available, Compute Engine creates your requested number of VMs. The Flex-start VMs run until you delete them, or until Compute Engine deletes the VMs at the end of their run duration. #### Ideal workloads for Flex-start Flex-start is ideal for workloads that can start at any time, such as the following: - Small model pre-training - Model fine-tuning - Simulations - Batch inference #### Characteristics of Flex-start Flex-start has the following characteristics: - You can request A4, A3 Ultra, A3 Mega, TPU7x, TPU v6e, and TPU v5p VMs. For TPU VMs, Cluster Director provisions all VMs within a slice together on a 2D or 3D interconnect topology; slice availability and dense allocation across nodes depend on resource availability. > [!NOTE] > **Note:** To create TPU7x Flex-start VMs, you must contact your account team or the sales team to request access. - You use the flex-start provisioning model to create Flex-start VMs, which has the following benefits: - You have a higher chance to obtain GPUs or TPUs. - You get a discount up to 53% for vCPUs, memory, GPUs, TPUs, and any attached Local SSD disks. - You can request resources for up to seven days. When your allocated resources reach the end of their maximum run duration, Cluster Director automatically requests new resources with the same run duration. > [!NOTE] > **Note:** If resources are unavailable at the end of a maximum run duration, then Cluster Director attempts to obtain resources to restart nodes for up to eight hours. After this time, Slurm sets the node state to `down`. The request remains active until Cluster Director allocates the required resources or you cancel the request. To cancel a pending request, modify a compute resource configuration, modify the number of nodes in a nodeset, or delete your cluster. #### How to use Flex-start To use Flex-start to create VMs in one or more of your cluster partitions, use one of the following methods: - [Create an AI-optimized cluster based on a template](https://docs.cloud.google.com/cluster-director/docs/create/cluster-from-template.md.txt) - [Create a custom cluster](https://docs.cloud.google.com/cluster-director/docs/create/custom-cluster.md.txt) ### Use Spot To run fault-tolerant workloads, you can obtain compute resources immediately based on availability. You get resources at the lowest price possible. However, Compute Engine can preempt Spot VMs at any time to reclaim capacity. #### Ideal workloads for Spot Spot is ideal for workloads where interruptions are acceptable, such as the following: - Batch processing - High performance computing (HPC) - Continuous integration and continuous deployment (CI/CD) - Data analytics - Media encoding - Online inference #### Characteristics of Spot Spot has the following characteristics: - You can create A4, A3 Ultra, A3 Mega, TPU7x, TPU v6e, TPU v5p, and N2 VMs. For TPU VMs, Cluster Director provisions all VMs within a slice together on a 2D or 3D interconnect topology; slice availability and dense allocation across nodes depend on resource availability. > [!NOTE] > **Note:** To create TPU7x Spot VMs, you must contact your account team or the sales team to request access. - You can immediately create clusters. Unless Compute Engine preempts the VMs to reclaim capacity, GPU and N2 VMs run until you stop or delete them, and TPU VMs run until you delete them. If you use GPU or N2 Spot VMs in a managed compute cluster, then you can only configure Compute Engine to stop (`TERMINATE`) them on preemption. - You use the spot provisioning model to create Spot VMs, which has the following benefits: - You have a higher chance to obtain GPUs or TPUs. - You get a discount up to 91% for vCPUs, memory, and any attached GPUs, TPUs, and Local SSD disks. #### How to use Spot To use Spot to create VMs in one or more of your cluster partitions, complete the following steps: 1. **View resource availability**. You can view the availability of A4, A3 Ultra, A3 Mega, and N2 machine types in the region or zone where you want to create Spot VMs. This action helps reduce the time that Cluster Director takes to allocate resources and create Spot VMs. For instructions, see [View resource availability for Spot VMs](https://docs.cloud.google.com/compute/docs/instances/view-vm-availability). 2. **View preemption rate and pricing**. You can view the historical and current preemption rate and pricing across A4, A3 Ultra, A3 Mega, and N2 machine types and zones. This information helps you choose the machine type and location that best fit your workload and budget. For instructions, see [View the preemption rate and pricing for Spot VMs](https://docs.cloud.google.com/compute/docs/instances/view-spot-preemption-price). 3. **Create your cluster**. To create your cluster with one or more partitions that use Spot VMs, select one of the following methods: - [Create an AI-optimized cluster based on a template](https://docs.cloud.google.com/cluster-director/docs/create/cluster-from-template.md.txt) - [Create a custom cluster](https://docs.cloud.google.com/cluster-director/docs/create/custom-cluster.md.txt) ### Use On-demand To run TPU workloads without a reservation or cluster components without GPU or TPU acceleration (such as login nodes), you can obtain on-demand resources based on availability at standard pricing. #### Ideal workloads for On-demand On-demand is ideal for the following workloads: - TPU training and inference workloads - Login nodes - CPU-bound computational tasks - General-purpose HPC #### Characteristics of On-demand On-demand has the following characteristics: - You can create C4A, N2, TPU7x, TPU v6e, and TPU v5p VMs. Cluster Director provisions TPU VMs in a slice together on a 2D or 3D interconnect topology. For C4A and N2 VMs, dense allocation depends on resource availability. You can use C4A VMs only for login nodes. - You can immediately create clusters. C4A and N2 VMs run until you stop or delete them, and TPU VMs run until you delete them. - You use the standard provisioning model, which is the default model. #### How to use On-demand To use On-demand to create VMs in one or more of your cluster partitions, use one of the following methods: - [Create an AI-optimized cluster based on a template](https://docs.cloud.google.com/cluster-director/docs/create/cluster-from-template.md.txt) - [Create a custom cluster](https://docs.cloud.google.com/cluster-director/docs/create/custom-cluster.md.txt)