Network services for deployments

The network services you configure depend on your chosen deployment option (VMs or clusters) and infrastructure.

This document is intended for architects, network engineers, and developers who want to understand the network services for their AI Hypercomputer deployments. This document assumes you have basic familiarity with cloud networking and distributed computing concepts. For more information about deployment options, see Deployment options overview.

For detailed information about network architecture, protocols, and hardware topologies, see the GPU networking overview.

Networking for AI-optimized GKE deployments with default configurations

When you create an AI-optimized GKE cluster with default settings, the Cluster Toolkit blueprint configures the network as follows:

  • The blueprint uses the default VPC network for the main GKE cluster.
  • It creates two additional VPCs: one for a second host Network Interface Card (NIC) and another for GPU-to-GPU remote direct memory access (RDMA) traffic.
  • The blueprint applies a pre-configured, Google-managed network profile to the VPC used for GPU traffic. This profile automatically configures the network for high-speed, low-latency RDMA performance.
  • The blueprint automatically creates eight dedicated subnets within the RDMA VPC, one for each RDMA NIC on an accelerator VM.
  • It configures firewall rules that allow all TCP, UDP, and ICMP traffic between nodes within the cluster.

Networking for GKE deployments with custom configuration

The network configuration for custom GKE setups depends on your workload type and infrastructure:

  • For General GPUs or non-distributed workloads, create a GKE cluster without GPUDirect RDMA. This configuration uses a single VPC network for all communication.
  • For Clustered GPUs or distributed workloads, create a GKE cluster with GPUDirect RDMA enabled. This configuration uses a multi-VPC environment that separates general-purpose traffic from high-bandwidth GPU-to-GPU communication.

Networking for Slurm cluster deployments

Use the Cluster Toolkit to deploy AI and ML workloads through customizable blueprints, such as for the A4 or A3 Ultra series.

The network components configured by the blueprint for Clustered GPU Slurm deployments are as follows:

  • The blueprint creates three distinct VPCs: a primary VPC for the Slurm control plane, a secondary VPC for general host-level traffic, and a dedicated high-performance VPC for GPU-to-GPU communication.
  • For the GPU data plane, the blueprint applies a pre-configured, Google-managed network profile optimized for RoCE.
  • The blueprint sets a dedicated IP address range required by the Filestore service, which provides the shared /home directory for the cluster.
  • It creates a temporary, isolated image-build VPC used only during the process of building the custom VM image for the cluster nodes.

Networking for Compute Engine instances

You can use Compute Engine to create standalone VMs, bulk compute instances, and managed instance groups (MIGs). The specific network requirements depend on the infrastructure model of the machine type:

  • Clustered GPUs: these machine series (A4X Max, A4X, A4, A3 Ultra, A3 Mega, and A3 High with 8 GPUs) require a multi-VPC network configuration to separate general host-to-host traffic from high-bandwidth GPU-to-GPU communication.
  • General GPUs: these machine series (G2, G4, A2, and N1 with T4 or V100) use a single-VPC architecture over gVNIC interfaces for all communication. A3 Edge is an exception that requires four data VPCs and GPUDirect-TCPX.

For detailed information about the NICs and network configuration for your machine type, see Networking and GPU machines.

What's next