Configure Tier 1 networking for a VM

This document describes how to configure Tier 1 networking performance for certain virtual machine (VM) instances in Google Distributed Cloud (GDC) air-gapped. VMs with Tier 1 networking configurations are especially useful for large, distributed compute workloads with lots of heavy internode communications, such as high performance computing (HPC), machine learning (ML), and deep learning (DL).

Before you begin

To use gdcloud command-line interface (CLI) commands, ensure that you have downloaded, installed, and configured the gdcloud CLI. All commands for Distributed Cloud use the gdcloud or kubectl CLI, and require an operating system (OS) environment.

Get the kubeconfig file path

To run commands against the Management API server, ensure you have the following resources:

Sign in and generate the kubeconfig file for the Management API server if you don't have one.
Use the path to the kubeconfig file of the Management API server to replace MANAGEMENT_API_SERVER in these instructions.

Request permissions and access

To perform the tasks listed in this page, you must have the Project VirtualMachine Admin role. Follow the steps to verify that you have the Project VirtualMachine Admin (project-vm-admin) role in the namespace of the project where the VM resides.

For VM operations using the GDC console or the gdcloud CLI, request your Project IAM Admin to assign you both the Project VirtualMachine Admin role and the Project Viewer (project-viewer) role.

Limitations

Tier 1 networking is supported with N2, N3, N4, A2, and A3 machine types that have at least 4 vCPUs.

Bandwidth tiers

The ingress and egress bandwidth limit represents the maximum possible amount of data per unit of time, such as gigabits per second (Gbps), that GDC allows a VM instance to transmit or receive from its network interfaces (NICs). The bandwidth includes data transferred between VM instances and object storage.

GDC has the following bandwidth limits:

The default bandwidth limit is 10 Gbps.
Tier 1 networking increases the maximum egress bandwidth limit for compute instances. The maximum egress bandwidth limit ranges from 16 Gbps to 25 Gbps, depending on the size and machine type of your instance.
The actual egress bandwidth is always less than or equal to the egress bandwidth limit.

To achieve the highest possible egress bandwidth, all of the following must be true:

The sending and receiving compute instances must be in the same zone.
Packets sent between the instances must use internal IP address destinations.
The number of TCP flows must be between 4 and 128 flows.
The sending instances must have TCP Segmentation Offload (TSO) enabled, and the receiving instances must have Generic Receive Offload (GRO) enabled.

N2 and N3 VMs

vCPUs	Internal IP	Tier 1 Internal IP
2	10 Gbps	Not applicable
4	10 Gbps	Not applicable
8	10 Gbps	16 Gbps
12	10 Gbps	25 Gbps
16	10 Gbps	25 Gbps
32	10 Gbps	25 Gbps
64	10 Gbps	25 Gbps

N4 VMs

vCPUs	Internal IP	Tier 1 Internal IP
4	10 Gbps	Not applicable
8	10 Gbps	Not applicable
16	10 Gbps	25 Gbps
32	10 Gbps	25 Gbps
48	10 Gbps	25 Gbps
64	10 Gbps	25 Gbps
80	10 Gbps	25 Gbps
160	10 Gbps	25 Gbps

A2 VMs

vCPUs	Internal IP	Tier 1 Internal IP
12	10 Gbps	18 Gbps
24	10 Gbps	25 Gbps

A3 VMs

vCPUs	Internal IP	Tier 1 Internal IP
28	10 Gbps	25 Gbps
56	10 Gbps	25 Gbps
112	10 Gbps	25 Gbps

Configure a VM with Tier 1 networking

Create instances and containers that use Tier 1 networking:

gdcloud

gdcloud compute instances create VM_NAME \
    --machine-type=MACHINE_TYPE \
    --image=BOOT_DISK_IMAGE_NAME --image-project=vm-system \
    --boot-disk-size=BOOT_DISK_SIZE \
    --network-performance-tier=tier_1

Replace the following:

Variable	Definition
`VM_NAME`	The name of the new VM. The name must only contain alphanumeric characters and dashes, and be no longer than 53 characters.
`MACHINE_TYPE`	The predefined machine type for the new VM. Select an available machine type: `gdcloud compute machine-types list`
`BOOT_DISK_IMAGE_NAME`	The name of the image to use for the new VM boot disk.
`BOOT_DISK_SIZE`	The size of the boot disk, such as `20GB`. This value must always be greater than or equal to the `minimumDiskSize` of the boot disk image.

API

kubectl --kubeconfig MANAGEMENT_API_SERVER \
    apply -n PROJECT -f - <<EOF
apiVersion: virtualmachine.gdc.goog/v1
kind: VirtualMachineDisk
metadata:
  name: VM_BOOT_DISK_NAME
spec:
  source:
    image:
      name: BOOT_DISK_IMAGE_NAME
      namespace: vm-system
  size: BOOT_DISK_SIZE
---
apiVersion: virtualmachine.gdc.goog/v1
kind: VirtualMachine
metadata:
  name: VM_NAME
spec:
  compute:
    virtualMachineType: MACHINE_TYPE
  disks:
    - virtualMachineDiskRef:
        name: VM_BOOT_DISK_NAME
      boot: true
      autoDelete: BOOT_DISK_AUTO_DELETE
  network:
    networkPerformanceTier: Tier_1
EOF