Google Cloud is focused on delivering world-class artificial intelligence (AI)
infrastructure to power your most demanding GPU-accelerated workloads across a
wide range of segments. You can use GPUs on Google Cloud to run AI, machine
learning (ML), scientific, analytics, engineering, consumer, and enterprise
applications.

Through our partnership with NVIDIA, Google Cloud delivers the latest GPUs while
optimizing the software stack with a wide array of storage and networking
options. For a full list of GPUs available, see [GPU platforms](https://docs.cloud.google.com/compute/docs/gpus.md.txt).

The following sections outline the benefits of GPUs on Google Cloud.

## GPU-accelerated VMs

On Google Cloud, you can access and provision GPUs in the way that best suits
your needs. A specialized [accelerator-optimized machine family](https://docs.cloud.google.com/compute/docs/accelerator-optimized-machines.md.txt) is
available, with pre-attached GPUs and networking capabilities that are ideal for
maximizing performance. These are available in the A4X Max, A4X, A4, A3, A2, G4,
and G2 machine series.

## Multiple provisioning options

You can provision clusters by using the accelerator-optimized machine family
with any of the following open-source or Google Cloud products.

### Vertex AI

Vertex AI is a fully-managed machine learning (ML) platform that you
can use to train and deploy ML models and AI applications. In Vertex AI
applications, you can use GPU-accelerated VMs to improve performance in the
following ways:

- [Use GPU-enabled VMs](https://docs.cloud.google.com/vertex-ai/docs/training/configure-compute) in custom training GKE worker pools.
- Use [open source LLM models from the Vertex AI Model Garden](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/open-models/use-open-models).
- Reduce [prediction](https://docs.cloud.google.com/vertex-ai/docs/predictions/configure-compute#gpus) latency.
- Improve performance of [Vertex AI Workbench](https://docs.cloud.google.com/vertex-ai/docs/workbench/instances/change-machine-type) notebook code.
- Improve performance of a [Colab Enterprise runtime](https://docs.cloud.google.com/colab/docs/create-runtime-template.md.txt).

### AI Hypercomputer

AI Hypercomputer is a supercomputing system that is optimized to
support your artificial intelligence (AI) and machine learning (ML) workloads.
It's an integrated system of performance-optimized hardware, open software, ML
frameworks, and flexible consumption models. AI Hypercomputer features
and services that are designed to let you deploy and manage large
numbers, up to tens of thousands, of accelerator and networking resources that
function as a single homogeneous unit. This option is ideal for creating a
densely allocated, performance-optimized infrastructure that supports
Google Kubernetes Engine (GKE), Slurm schedulers, and managed compute clusters. For
more information, see the [AI Hypercomputer overview](https://docs.cloud.google.com/ai-hypercomputer/docs/overview.md.txt).

To get started with AI Hypercomputer, see
[Choose an orchestrator and deployment option](https://docs.cloud.google.com/ai-hypercomputer/docs/choose-platform.md.txt).

### Compute Engine

You can also create and manage individual VMs or small clusters of VMs with
attached GPUs on Compute Engine. This method is mostly used for running
graphics-intensive workloads, simulation workloads, or small-scale ML model
training.

The following table shows the methods that you can use to create VMs that have
GPUs attached:

|---|---|
| **Deployment option** | **Deployment guides** |
| Create a VM for serving and single node workloads | [Create an A3 Edge or A3 High VM](https://docs.cloud.google.com/compute/docs/gpus/gpudirect.md.txt) |
| Create managed instance groups (MIGs) This option uses the [Dynamic Workload Scheduler (DWS)](https://cloud.google.com/blog/products/compute/introducing-dynamic-workload-scheduler). | [Create a MIG with GPU VMs](https://docs.cloud.google.com/compute/docs/instance-groups/create-mig-with-gpu-vms.md.txt) |
| Create VMs in bulk | [Create a group of GPU VMs in bulk](https://docs.cloud.google.com/compute/docs/gpus/create-gpu-vm-bulk.md.txt) |
| Create a single VM | [Create a single GPU VM](https://docs.cloud.google.com/compute/docs/gpus/create-vm-with-gpus.md.txt) |
| Create virtual workstations | [Create a virtual GPU-accelerated workstation](https://docs.cloud.google.com/compute/docs/virtual-workstation.md.txt#choose_a_virtual_workstation) |

### Cloud Run

You can configure GPUs for your Cloud Run instances. GPUs are ideal for
running AI inference workloads using large language models on Cloud Run.

On Cloud Run, consult these resources for running AI workloads on GPUs:

- [Configure GPUs for a Cloud Run service](https://docs.cloud.google.com/run/docs/configuring/services/gpu.md.txt)
- [Load large ML models on Cloud Run with GPUs](https://docs.cloud.google.com/run/docs/configuring/services/gpu-best-practices.md.txt#model-loading-recommendations)
- [Tutorial: Run LLM inference on Cloud Run GPUs with Ollama](https://docs.cloud.google.com/run/docs/tutorials/gpu-gemma2-with-ollama)