You can enable and manage graphics processing unit (GPU) resources on your
containers. For example, you might prefer running artificial intelligence (AI)
and machine learning (ML) notebooks in a GPU environment. GPU support
is enabled by default in Google Distributed Cloud (GDC) air-gapped appliance.

## Before you begin

To complete the tasks in this document, you must request the necessary
permissions and prepare your environment.

### Request IAM roles

To create, delete, edit, or view the resources in a Kubernetes cluster, ask your
Organization IAM Admin to grant you the **Namespace Admin** (`namespace-admin`)
role in your project namespace. This role lets you deploy GPU workloads in your
project namespace.

### Prepare your environment

- [Sign in and generate](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/application/pa-user/iam/sign-in#kubernetes-cluster-kubeconfig)
  the kubeconfig file for the bare metal Kubernetes cluster.

- Use the kubeconfig path of the Kubernetes cluster to replace
  `CLUSTER_KUBECONFIG` in these instructions.

## Configure a container to use GPU resources

To use GPUs in a container, complete the following steps:

1. Confirm your Kubernetes cluster nodes support your GPU resource allocation:

       kubectl describe nodes NODE_NAME

   Replace `NODE_NAME` with the node managing the GPUs
   you want to inspect.

   The relevant output is similar to the following snippet:

       Capacity:
         nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE: 1
       Allocatable:
         nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE: 1

2. Add the `.containers.resources.requests` and `.containers.resources.limits`
   fields to your container spec. Since your Kubernetes cluster is preconfigured
   with GPU machines, the configuration is the same for all workloads:

        ...
        containers:
        - name: CONTAINER_NAME
          image: CONTAINER_IMAGE
          resources:
            requests:
              nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE: 1
            limits:
              nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE: 1
        ...

   Replace the following:
   - `CONTAINER_NAME`: the name of the container.
   - `CONTAINER_IMAGE`: the container image to access the GPU machines. You must include the container registry path and version of the image, such as `REGISTRY_PATH/hello-app:1.0`.
3. Containers also require additional permissions to access GPUs. For each
   container that requests GPUs, add the following permissions to your
   container spec:

       ...
       securityContext:
        seLinuxOptions:
          type: unconfined_t
       ...

4. Apply your container manifest file:

       kubectl apply -f CONTAINER_MANIFEST_FILE \
           -n NAMESPACE \
           --kubeconfig CLUSTER_KUBECONFIG

   Replace the following:
   - `CONTAINER_MANIFEST_FILE`: the YAML file for your container workload custom resource.
   - `NAMESPACE`: the project namespace in which to deploy the container workloads.
   - `CLUSTER_KUBECONFIG`: the kubeconfig file for the bare metal Kubernetes cluster to which you're deploying container workloads.
5. Verify that your pods are running and are using the GPUs:

       kubectl get pods -A | grep CONTAINER_NAME \
           -n NAMESPACE \
           --kubeconfig CLUSTER_KUBECONFIG

   The relevant output is similar to the following snippet:

       Port:           80/TCP
       Host Port:      0/TCP
       State:          Running
       Ready:          True
       Restart Count:  0
       Limits:
         nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE:  1
       Requests:
         nvidia.com/gpu-pod-NVIDIA_A100_80GB_PCIE:  1