> [!WARNING]
>
> **Preview**
>
>
> This product or feature is
>
> subject to the "Pre-GA Offerings Terms" in the General Service Terms section
> of the [Service Specific
> Terms](https://docs.cloud.google.com/terms/service-terms#1).
>
> Pre-GA products and features are available "as is" and might have limited support.
>
> For more information, see the
> [launch stage descriptions](https://cloud.google.com/products/#product-launch-stages).

<button class="button button-with-icon cloud-select-dropdown__button button-primary" aria-haspopup="true">Version 1.13 </button>

- [Latest](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/latest/vm-runtime/use-gpu)
- [Version 1.16](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.16/vm-runtime/use-gpu.md.txt)
- [Earlier versions (unsupported)](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.13/getting-support.md.txt#version-support)

<br />

This document shows you how to enable ​​NVIDIA® GPU support for virtual
machines (VMs) that run using VM Runtime on Google Distributed Cloud. You learn how to
install the NVIDIA drivers on your Google Distributed Cloud nodes, verify that the
GPUs are available, and assign GPUs to VMs.

## Before you begin

To complete this document, you need access to the following resources:

- Access to Google Distributed Cloud version 1.12.0 (`anthosBareMetalVersion: 1.12.0`) or higher cluster. You can use any cluster type capable of running workloads. If needed, [try Google Distributed Cloud on Compute Engine](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.13/try/gce-vms.md.txt) or see the [cluster creation overview](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.13/installing/creating-clusters/create-clusters-overview.md.txt).
- The `virtctl` client tool installed as a plugin for `kubectl`. If needed, [install the virtctl client tool](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.13/vm-runtime/quickstart.md.txt#install_the_virtctl_client_tool).

## Supported Nvidia GPU cards

Google Distributed Cloud version 1.13 or higher support the following NVIDIA GPUs:

- Tesla T4
- Tesla P4
- Tesla V100 SXM2 32 GB
- A100 SXM4 40 GB
- A100 PCIe 40 GB
- A100 SXM4 80 GB
- A100 PCIe 80 GB

## Install NVIDIA drivers on nodes

Before your VMs can use the NVIDIA GPUs, you must configure your
Google Distributed Cloud nodes to support the GPU devices. To install the NVIDIA
drivers on your nodes, complete the following steps on each node in your cluster
that includes an NVIDIA GPU. This document uses a
[supported Ubuntu version](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.13/installing/os-reqs.md.txt)
for the nodes:

1. Connect to your Google Distributed Cloud node that you want to configure for GPU support.
2. Get the kernel version of your node:

   ```bash
   KERNEL_VERSION="$(uname -r)"
   ```
3. Update your Ubuntu node and install the appropriate kernel headers:

   ```bash
   sudo apt update && \
   apt install -y linux-headers-${KERNEL_VERSION}
   ```
4. Install the `build-essential` package so that you can compile the Nvidia
   drivers in a following step:

   ```bash
   sudo apt install -y build-essential
   ```
5. Download the appropriate NVIDIA driver package for your GPU. For a complete
   list of drivers, see
   [NVIDIA Driver Downloads](https://www.nvidia.com/download/index.aspx).

   The following example downloads the `Linux x86_64` version `470.82.01` driver:

   ```bash
   wget https://us.download.nvidia.com/tesla/470.82.01/NVIDIA-Linux-x86_64-470.82.01.run
   ```
6. Install the NVIDIA driver package. Use the name of the NVIDIA driver package
   you downloaded in the previous step:

   ```bash
   sudo sh NVIDIA-Linux-x86_64-470.82.01.run \
     --accept-license \
     --silent \
     --no-nouveau-check
   ```
7. Load the NVIDIA kernel module:

   ```bash
   sudo modprobe nvidia
   ```
8. Repeat the steps in this section on each node in your cluster that has a
   NVIDIA GPU.

## Enable GPU support in VM Runtime on Google Distributed Cloud

After you install the NVIDIA drivers on your Google Distributed Cloud node(s), you
enable GPU support in VM Runtime on Google Distributed Cloud. Your VMs can then access the
GPUs on the nodes.

Each node reboots as part of the following process. Your VMs may be affected by
this reboot process. If possible and configured to do so, migratable VMs migrate
to other nodes. For more information, see how to
[configure the eviction policy for VMs during maintenance events](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.13/vm-runtime/eviction-policy.md.txt).

To enable GPU support in VM Runtime on Google Distributed Cloud, complete the following
steps.

1. Edit the `VMRuntime` custom resource:

   ```bash
   kubectl edit vmruntime vmruntime
   ```
2. Add the `enableGPU: true` property to the `VMRuntime` manifest:

   ```yaml
   apiVersion: vm.cluster.gke.io/v1
   kind: VMRuntime
   metadata:
     name: vmruntime
   spec:
     enabled: true
     enableGPU: true
   ...
   ```
3. Save and close the `VMRuntime` custom resource in your editor.

4. Check the status of the GPU controllers in the `vm-system` namespace:

   ```bash
   kubectl get pods --namespace vm-system  -w
   ```

   It takes about five minutes for the controllers to be successfully enabled.
   Wait for the `STATUS` to show
   `Running` for all the GPU controllers. The following example output shows the
   desired state:

   ```bash
   NAME                                          READY  STATUS    RESTARTS     AGE
   gpu-controller-controller-manager-gwvcb       2/2    Running   0            10m
   kubevirt-gpu-dp-daemonset-2lfkl               1/1    Running   0            10m
   kubevm-gpu-driver-daemonset-5fwh6             1/1    Running   0            10m
   nvidia-gpu-dp-daemonset-9zq2w                 1/1    Running   0            10m
   nvidia-mig-manager-5g7pz                      1/1    Running   0            10m
   vm-controller-controller-manager-7b6df6979b   2/2    Running   2 (13m ago)  14m
   ```
5. Verify that the GPUs are available for use when the GPU controllers all
   report their status as `Running`:

   ```bash
   kubectl get gpuallocations --namespace vm-system
   ```

   The following example output shows that the GPUs on the nodes are available
   for use. Each node in your cluster with GPU support is shown. You allocate
   them to VMs in the next section:

   ```bash
   NAME       ALLOCATED   DEVICEMODEL
   bm-node1   true        Tesla A100 SXM4 40GB
   bm-node2   true        Tesla A100 SXM4 40GB
   ```

## Allocate GPUs for use with VMs

With GPU support configured in the Anthos clusters on bare metal nodes and in
VM Runtime on Google Distributed Cloud, allocate the GPUs for use with VMs. By default,
GPUs are allocated for use with pods (containers).

1. Edit the `GPUAllocation` custom resource for use with VMs. This step assigns
   the GPUs on the nodes for use with VMs:

   ```bash
   kubectl edit gpuallocation NODE_NAME --namespace vm-system
   ```

   Replace `NODE_NAME` with the name of your node that you want to allocate GPUs
   from.
2. Configure how many GPUs to allocate to VMs. Initially, all GPUs are allocated
   to pods.

   The total number of GPUs allocated to VMs and pods must equal the number of
   GPUs in the node. For example, you might have four GPUs in your node. If you
   allocate two GPUs to VMs, then two GPUs remain allocated to pods. The
   `GPUAllocation` manifest is rejected if you try to allocate two GPUs to VMs
   and one GPU to pods, as one GPU is left unallocated.

   Update the number of GPUs on the node that you want to allocate for use with
   VMs, as shown in the following example:

   ```yaml
   apiVersion: gpu.cluster.gke.io/v1
   kind: GPUAllocation
   metadata:
     name: gpu-w2
     namespace: vm-system
   spec:
     node: gpu-w2
     pod: 0
     vm: 4
   ```

   In this example, all four GPUs installed in the node are allocated to VMs. No
   GPUs are allocated to pods.
3. Save and close the `GPUAllocation` custom resource in your editor.

4. Verify that the GPUs report their `ALLOCATED` status as `true`:

   ```bash
   kubectl get gpuallocations --namespace vm-system
   ```

   The following example output shows that the GPUs on the nodes are available
   for use:

   ```bash
   NAME     ALLOCATED   DEVICEMODEL
   gpu-w1   true        Tesla A100 SXM4 40GB
   gpu-w2   true        Tesla A100 SXM4 40GB
   ```

## Create a VM with GPU support

You can now create a VM that uses the GPU from the node. In the VM custom
resource, you specify the name and quantity of GPUs to allocate from the node.

1. Get the name of the GPU card from the host:

   ```bash
   kubectl describe node NODE_NAME
   ```

   Replace `NODE_NAME` with the name of the host that you want
   to get the GPU name from.

   The following example output shows that the allocatable GPU name on this node
   is `NVIDIA_A100_SXM4_40GB`:

   ```bash
   Name:               bm-node1
   Roles:              worker
   [...]
   Allocatable:
     cpu:                                         47810m
     [...]
     memory:                                      336929400Ki
     nvidia.com/gpu-vm-NVIDIA_A100_SXM4_40GB:     1
   [...]
   ```
2. Create a `VirtualMachine` manifest, such as `my-gpu-vm.yaml`, in the editor
   of your choice:

   ```bash
   nano my-gpu-vm.yaml
   ```
3. Copy and paste the following YAML manifest:

   ```yaml
   apiVersion: vm.cluster.gke.io/v1
   kind: VirtualMachine
   metadata:
     name: VM_NAME
   spec:
     interfaces:
       - name: eth0
         networkName: pod-network
         default: true
     disks:
       - virtualMachineDiskName: VM_NAME-boot-dv
         boot: true
         gpu:
           model: nvidia.com/gpu-vm-GPU_NAME
           quantity: 1
   ```

   In this YAML file, define the following settings:
   - `VM_NAME`: the name for your VM.
   - `GPU_NAME`: the GPU name from the node to allocate to the VM.
     - This GPU name is shown in the output of the `kubectl describe
       node`command from a previous step, such as `NVIDIA_A100_SXM4_40GB`.

   The VM connects `eth0` to the default `pod-network` network.

   The boot disk named `VM_NAME-boot-dv`
   must already exist. For more information, see
   [Create and manage virtual disks](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.13/vm-runtime/create-manage-disks.md.txt).
4. Save and close the VM manifest in your editor.

5. Create the VM using `kubectl`:

   ```bash
   kubectl apply -f my-gpu-vm.yaml
   ```
6. When your VM is running,
   [connect to the VM](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.13/vm-runtime/connect-vms.md.txt)
   and verify that the GPU hardware is available.

## What's next

- [Edit a VM in Google Distributed Cloud](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.13/vm-runtime/edit-vm.md.txt).