- AI infrastructure resources colocation
- Cluster topology-aware placement
- Cluster operational mode
- Cluster maintenance scheduling and controls
- Cluster monitoring and diagnostic tooling
Most cluster management capabilities are only supported for reservation-bound capacity—infrastructure that uses the reservation-bound provisioning model, which only supports clustered GPUs machine types. For capacity that uses any other provisioning model, only the following cluster management capabilities are available:
- AI infrastructure resources colocation
- Cluster topology-aware placement
- Manage host events across instances
AI infrastructure resources colocation
When you use clustered GPUs, such as A4X Max, A4X, A4, A3 Ultra, A3 Mega, and A3 High (8 GPUs), you can request host machines that Compute Engine provisions as close together as possible. These machines offer the following features:
Compute Engine provisions the machines as blocks of resources.
A dynamic machine learning (ML) network fabric interconnects the machines.
This resource arrangement minimizes network hops and optimizes for lowest network latency. To deploy blocks of accelerator-optimized machines that Compute Engine densely allocates, you can use the following options:
Resources are required to be densely allocated with each other when you do one or more of the following:
Create resources that use the same Future reservation in AI Hypercomputer or Future reservation in calendar mode.
Deploy a managed instance group (MIG) that uses resize requests and create the resources in the same request.
Specify that resources use the same compact placement policy or workload policy that specifies a max distance value. Specifically, resources are only required to be as close as the max distance value of the policy specifies.
Resources are optionally densely allocated with each other, based on best-effort availability, when you do one or more of the following:
Create resources in the same request for Flex-start VMs.
Specify that resources use the same compact placement policy or workload policy that doesn't specify a max distance.
Cluster topology-aware placement
After you create clustered GPUs, you can get topology information at the node and cluster levels. This information helps you do the following:
Adjust your application or workload design to further minimize network latency.
Understand and troubleshoot network latency and performance issues for instances that communicate frequently with each other. These issues can occur if the instances are unexpectedly located far apart.
Specifically, topology information is supported as follows:
Topology information works best with reservation-bound capacity, which is required to view the topology of a reservation.
For Spot VMs, topology information only appears when a machine uses a compact placement policy or workload policy.
For more information, see the following:
- For instances and Slurm clusters, see View compute instances topology.
- For GKE clusters, see View GKE node topology and Schedule GKE workloads with Topology Aware Scheduling (TAS).
Cluster operational mode
When you reserve reservation-bound capacity to create compute instances or clusters, the machine type that you reserve determines the cluster operational mode for the instances. This mode specifies how your instances behave after host errors or faulty host reports. The available operational modes for an instance are managed mode, where Compute Engine automatically replaces any faulty machines but holds back part of your reserved capacity to provide your instances have the necessary resources to restart. Or all-capacity mode, where you have access to your full reserved capacity but are responsible for managing failures and planned maintenance.
For more information, see Reservation operational mode.
Cluster maintenance scheduling and controls
You control maintenance of clustered GPUs by using topology-aware scheduling in a block of resources. This capability helps synchronize upgrades so that your workloads are more resilient to host events and minimize disruptions. This approach helps improve the goodput of your workload.
To facilitate full control of maintenance events, you can use the following features:
Maintenance scheduling type
When you reserve reservation-bound capacity to create compute instances or clusters of clustered GPUs, you can define how Compute Engine maintains the infrastructure that your instances run on. Based on the machine type that you want to use for your instances, you can choose between synchronized maintenance across instances (grouped), or different maintenance schedules (independent).
For more information, see Maintenance scheduling types.
Manage host events
After you create clustered GPUs and start your workload, you can set up alerts and receive notifications when maintenance for your instances or reserved blocks is scheduled, starts, or is completed. You can also view and, if needed, manually start maintenance on an instance or reserved block before its scheduled time. These options help you proactively control and minimize downtimes to your workloads.
For more information, see the following:
Cluster monitoring and diagnostic tooling
For monitoring and troubleshooting, clustered GPU machine types include the following services for reservation-bound capacity:
VM health degradation prediction, which helps you identify VMs that are likely to degrade within the next five hours.
Faulty host reporting, which you can use to flag issues with individual host machines.
Support for Cloud Monitoring metrics, which help you monitor networks and GPUs performance.