Cluster creation process overview

This document explains how to plan and configure the infrastructure for a cluster in Cluster Director. Before you create a cluster for your artificial intelligence (AI), machine learning (ML), or high performance computing (HPC) workload, you make two foundational choices:

• Cluster orchestrator: choose between Slurm orchestration or managed compute orchestration. Slurm provides a built-in job scheduler, login nodes, a controller node, and autoscaled partitions. Managed compute orchestration instead uses zonal managed instance groups (MIGs) that you manage with your own orchestrator or by running your workloads directly on the compute nodes. For a detailed comparison, see Choose an orchestrator.

• Creation method: you can either deploy a cluster from a pre-configured template that populates recommended settings for common GPU and TPU workloads, or create a custom cluster by configuring each resource step by step.

Cluster creation process

When you create a cluster, you define reusable compute, network, and storage resource configurations, and then group those resources in your chosen orchestrator. Follow these steps to plan your cluster configuration:

1. Configure compute resources

2. Configure networking resources

3. Configure storage resources

4. Configure your orchestrator

5. Optional: Configure data protection

If you want an interactive planning session before you configure resources, then run the following prompt in Compute Advisor:

agent_mode_2 Compute Advisor Compute Advisor is an AI-powered assistant that you can prompt to get customized recommendations for your infrastructure based on information such as your quota limits, existing reservations, and resource location constraints.

This prompt starts an interactive session to evaluate your project quotas and reservations, recommends machine and orchestrator configurations for your cluster, and outputs a summary table with recommended Google Cloud console settings.

Act as a cloud architect. Help me design a cluster in Cluster Director based on my project quotas, existing reservations, and available regions.

Start by asking me about my workload type (such as foundation model training, fine-tuning, or inference), my target scale, and whether I want to use the built-in Slurm scheduler or my own orchestrator on Compute Engine managed instance groups (MIGs). After I provide my workload details, walk me through the five setup steps in this guide:

1. Configure compute resources: Recommend machine series for compute nodes (A4X Max, A4X, A4, A3 Ultra, A3 Mega, TPU7x, TPU v6e, TPU v5p, or N2) and, for Slurm clusters, controller nodes (N2) and login nodes (C4A or N2), a deployment region and zone, and a consumption option (reservations, Flex-start, Spot, or on-demand).

2. Configure networking resources: Recommend whether to let Cluster Director create a new VPC network or to use an existing VPC or Shared VPC network with the required subnet, Private Google Access, health check (TCP port 8319), and IAP (TCP port 22) firewall rules.

3. Configure storage resources: Recommend new or existing Filestore instances, Google Cloud Managed Lustre instances, or Cloud Storage buckets (noting that Slurm clusters require a Filestore or Managed Lustre instance for the shared /home directory).

4. Configure the orchestrator:

7. Configure data protection (optional): Identify whether I need a VPC Service Controls service perimeter and list the required restricted APIs.

Format the recommendations as a Markdown summary table for each step, followed by recommended Google Cloud console settings and next steps.

Act as a cloud architect. Help me design a cluster in Cluster Director based on my project quotas, existing reservations, and available regions.

Start by asking me about my workload type (such as foundation model training, fine-tuning, or inference), my target scale, and whether I want to use the built-in Slurm scheduler or my own orchestrator on Compute Engine managed instance groups (MIGs). After I provide my workload details, walk me through the five setup steps in this guide:

1. Configure compute resources: Recommend machine series for compute nodes (A4X Max, A4X, A4, A3 Ultra, A3 Mega, TPU7x, TPU v6e, TPU v5p, or N2) and, for Slurm clusters, controller nodes (N2) and login nodes (C4A or N2), a deployment region and zone, and a consumption option (reservations, Flex-start, Spot, or on-demand).

2. Configure networking resources: Recommend whether to let Cluster Director create a new VPC network or to use an existing VPC or Shared VPC network with the required subnet, Private Google Access, health check (TCP port 8319), and IAP (TCP port 22) firewall rules.

3. Configure storage resources: Recommend new or existing Filestore instances, Google Cloud Managed Lustre instances, or Cloud Storage buckets (noting that Slurm clusters require a Filestore or Managed Lustre instance for the shared /home directory).

4. Configure the orchestrator:

7. Configure data protection (optional): Identify whether I need a VPC Service Controls service perimeter and list the required restricted APIs.

Format the recommendations as a Markdown summary table for each step, followed by recommended Google Cloud console settings and next steps.

Edit and run prompt in Compute Advisor

Configure compute resources

First, define the compute resource configurations for the Compute Engine instances in your cluster. Each Slurm nodeset or MIG uses a single compute resource configuration and machine type. You can create multiple compute resource configurations in the same cluster so that different Slurm nodesets or MIGs can use different machine types. Evaluate the following settings for each compute resource configuration:

• Machine series: choose the machine series and machine type for each node role in your cluster:

  • Compute nodes: for Slurm or managed compute clusters, choose a machine series based on your workload:

    • Foundational model training and inference: A4X Max or A4X machine series

    • Large model training, fine-tuning, and inference: A4, A3 Ultra, TPU7x, TPU v6e, or TPU v5p machine series

    • Mainstream model inference and fine-tuning: A3 Mega machine series

    • General-purpose HPC: N2 machine series

    For more guidance on accelerator selection, see Ideal use cases for GPU instances and Ideal use cases for TPU instances.

  • Controller nodes: for Slurm clusters only, choose the N2 machine series.

  • Login nodes: for Slurm clusters only, choose the C4A or N2 machine series.

  For the full list of supported machine types, see CPU machines, GPU machines, and TPU machines.

• Location: based on the locations where your chosen machine series are available, select the region and zone where you want to deploy your cluster. All compute, storage, and subnetwork resources in your cluster must exist within the same zone. Cluster Director doesn't support AI zones, which are zones with names that contain -ai after the region name. For more information, see Cluster Director locations.

• Consumption option: choose the consumption option that you want to use to obtain compute capacity (such as a reservation, Flex-start VMs, Spot VMs, or on-demand instances). Each option affects the availability, pricing, and lifecycle of your compute instances. For more information, see Choose a consumption option.

Configure networking resources

Next, choose how to configure the Virtual Private Cloud (VPC) network that connects the compute instances and shared storage in your cluster. You can choose one of the following options:

• Create a new VPC network

• Use an existing VPC or Shared VPC network

Create a new VPC network

If you choose to create a new VPC network, then Cluster Director creates the network and subnetwork on your behalf and configures the required firewall rules so that you can connect to your cluster.

Use an existing VPC or Shared VPC network

If you choose to use an existing VPC or Shared VPC network, then you must configure the network and subnetwork before you create the cluster:

• Enable Private Google Access in your network. For instructions, see Private Google Access configuration.

• Configure firewall rules that allow the following traffic:

  • SSH connections to VMs

  • Internal ingress connections among VMs

• Configure the subnetwork to meet the following requirements:

  • The subnetwork must allow health checks over the TCP protocol on port 8319. For more information, see Use health checks.

  • The subnetwork must have a firewall rule that allows Identity-Aware Proxy (IAP) to access the VMs over the TCP protocol on port 22. For more information, see Prepare your project for IAP TCP forwarding.

  • The subnetwork can't overlap with the following IPv4 address ranges:

    • 10.20.1.0/28

    • 10.0.0.0/17

    • 192.168.64.0/18

    • 192.168.128.0/18

• If you want to use a Shared VPC network, then you must also complete the following steps:

  • Configure private services access.

  • Enable the Cloud DNS API in the network's host project.

  • Grant the following Identity and Access Management (IAM) roles either at the host project level or at the subnetwork level:

    • Grant access to all subnets in the host project: grant the following IAM roles:

      • The Compute Network User (roles/compute.networkUser) role to the Google APIs Service Agent

      • The Hypercompute Shared VPC Service Agent (roles/hypercomputecluster.sharedVpcServiceAgent) role to the Cluster Director Service Agent ([e-mail])

    • Grant access to specific subnets: if your organization's policies require more granular control, then use this option. Grant the following IAM roles:

      • The Compute Network User (roles/compute.networkUser) role on the subnetwork to the Google APIs Service Agent and Cluster Director Service Agent ([e-mail])

      • The Compute Network Viewer (roles/compute.networkViewer) role on the network's host project to the Cluster Director Service Agent ([e-mail])

      • A custom role that includes the dns.managedZones.get, dns.managedZones.list, dns.networks.bindPrivateDNSZone, and dns.networks.targetWithPeeringZone permissions on the network's host project to the Cluster Director Service Agent ([e-mail])

  For more information about how to set up a Shared VPC network, see Set up Shared VPC.

Configure storage resources

After you plan your compute and networking resources, define the shared network storage that your cluster uses. For a comparison of storage options, see Storage in Cluster Director.

• Shared network storage requirements by orchestrator:

  • Slurm clusters: you must configure a Filestore instance or Google Cloud Managed Lustre instance for the shared /home directory across all login and compute nodes. You can also mount additional Filestore instances, Managed Lustre instances, or Cloud Storage buckets for datasets and checkpoints.

  • Managed compute clusters: shared network storage is optional. You can mount any combination of Filestore instances, Managed Lustre instances, or Cloud Storage buckets on your MIGs, or create a cluster without shared network storage.

• New or existing storage resources: when you add a Filestore instance, Managed Lustre instance, or Cloud Storage bucket to your cluster, you can choose one of the following options:

  • Let Cluster Director create a new storage resource with pre-configured settings when you create the cluster.

  • Use an existing storage resource. Existing Filestore and Managed Lustre instances must use the same VPC network as your cluster.

  If you use the Dynamic tier for Managed Lustre, then your cluster must use an existing VPC network that allocates the dynamic_tier_capacity quota.

  If you create new Cloud Storage buckets when you create your cluster, then you can specify only the Rapid storage class, Standard storage class, or Autoclass. If you want to use other storage classes, then you must update your buckets after you create them. For instructions, see Change object storage classes.

  Caution: You can't change the storage class of a Rapid bucket after you create it. Rapid buckets are zonal resources and must always use the Rapid storage class.

Note: You can use both new and existing storage resources in the same cluster. If you use an existing Filestore or Managed Lustre instance, then the instance must be on the same VPC network as your cluster. Each storage resource that you define must specify either a new resource configuration or an existing resource configuration, not both.

Configure your orchestrator

Next, configure your chosen orchestrator to organize your compute and storage resources into node groups and configure node boot disks. For a detailed comparison between Slurm orchestration and managed compute orchestration, see Choose an orchestrator.

Configure the Slurm orchestrator

When you configure a Slurm cluster, you define the following components:

• Login and controller nodes: specify the machine series and boot disk settings for the controller node and login nodes, which manage the Slurm scheduler and accept user connections:

  • Slurm controller version: when you create a cluster by using the Google Cloud CLI or the REST API, you can specify the Slurm controller version; you can't modify the controller version for an existing cluster. Compute nodes can't run a newer Slurm version than the controller node. For example, if the controller node in your cluster uses Slurm 25.05, then your compute nodes can't use an OS image that requires Slurm 26.05. To check the Slurm version of an OS image, see Supported OS images.

  • Boot disks and Google Cloud Hyperdisk pools: controller nodes require standalone boot disks. For login nodes and compute nodes, you can specify standalone boot disks or Hyperdisk pools if your selected machine series supports Hyperdisk. If you use storage pools for node boot disks, then you can use only existing storage pools or Hyperdisk Exapools that use Hyperdisk Balanced volumes. Storage pools must reside in the same project and zone as the nodes that use them.

• Partitions and nodesets: group your compute resource configurations into one or more Slurm partitions and nodesets so that you can route jobs to different pools of hardware. For each nodeset, specify the static node count (staticNodeCount), the maximum dynamic node count (maxDynamicNodeCount), and the node boot disk settings. Each nodeset and partition must meet the following requirements:

  • You can specify only one compute resource configuration for each nodeset in your cluster.

  • If you create A4X VMs in a nodeset, then the total number of A4X VMs in the nodeset, which is the sum of the static node count and the dynamic node count, must be a multiple of 18. Because Compute Engine provisions A4X capacity in indivisible sub-blocks of 18 VMs, nodesets can't share unused capacity.

  • If you create TPU instances in a nodeset, then you can configure either a static node count or a dynamic node count, not both. For a static TPU nodeset, you must specify an accelerator topology, and the static node count must be a multiple of the number of VMs per slice in that topology.

  • A Slurm partition that contains a TPU nodeset can't contain GPU or CPU nodesets, and can't combine static and dynamic TPU nodesets.

• Optional Slurm settings: you can also specify custom slurm.conf parameters, and startup (prologBashScripts) or shutdown (epilogBashScripts) scripts.

Configure managed compute orchestration (MIGs)

When you configure a managed compute cluster, you define one or more zonal Compute Engine MIGs with the following settings:

• Compute resource, target size, and TPU topology: each MIG references one compute resource configuration and specifies the target number of compute instances (targetSize) to maintain in the group. For a MIG with TPU instances, specify a TPU slice topology; each TPU MIG represents a single TPU slice, and the total number of TPU chips in the MIG must equal the total chip count of the slice topology.

• Boot disks and additional disks: for each MIG, choose a supported managed compute OS image or a custom image, configure the boot disk, and attach per-compute instance read-write additional disks.

• Compute instance access, startup scripts, and metadata: you can customize the service account and access scopes, enable or disable OS Login and external IP addresses, and attach startup scripts, custom metadata, and resource labels for each MIG.

Optional: Configure data protection

If you want to prevent unauthorized parties from copying sensitive data from your cluster, then use VPC Service Controls. Otherwise, skip this section. VPC Service Controls lets you control communication to and among services that Google manages. To use Cluster Director with VPC Service Controls, you create a security boundary around your resources, and Google creates a service perimeter to enforce this boundary.

To create a service perimeter, follow the instructions in Create a service perimeter and specify the following configurations:

• Perimeter resources (perimeterResources): include the project in which you want to create your cluster. If you want to use an existing Shared VPC network in your cluster, then you must also include the host project of that network.

• Restricted services (restrictedServices): include the following APIs:

  • Compute Engine API (compute.googleapis.com)

  • Filestore API (file.googleapis.com)

  • Hypercompute Cluster API (hypercomputecluster.googleapis.com)

  • Cloud Logging API (logging.googleapis.com)

  • Google Cloud Managed Lustre API (lustre.googleapis.com)

  • Cloud Monitoring API (monitoring.googleapis.com)

After you create the service perimeter, the changes can take up to 30 minutes to take effect. After that time, the service perimeter applies to any cluster that you create in your project.

What's next

• Try out Cluster Director

• Choose a consumption option