You can use GPU with your Cloud Run services for AI and other compute-intensive workloads. GPU works well for AI inference workloads like large language models (LLMs) or other compute-intensive, non-AI use cases such as video transcoding and 3D rendering. Google provides NVIDIA RTX PRO 6000 Blackwell GPU with 96 GB of GPU memory (VRAM) for a full GPU or 48 GB of GPU memory (VRAM) for a fractional (0.5) GPU, and NVIDIA L4 GPU with 24 GB of GPU memory (VRAM). GPU memory is separate from instance memory.
When using an LLM, be sure to also see Best practices: Cloud Run services with GPU.
GPU on Cloud Run is fully managed, with no extra drivers or libraries required. Cloud Run GPU offers on-demand availability with no reservations needed, similar to the way on-demand CPU and on-demand memory work in Cloud Run. To save on cost, Cloud Run services that use GPU can scale down to zero when not in use.
Cloud Run container instances with an NVIDIA RTX PRO 6000 Blackwell Blackwell or L4 GPU start in about 5 seconds. Your container can use the GPU as soon as the startup finishes.
You can configure up to one GPU per Cloud Run container instance. If you use sidecar containers, the GPU can only be attached to one container.
Supported GPU types
Cloud Run supports two GPU types:
NVIDIA RTX PRO 6000 Blackwell GPU:
- Full GPU with the current NVIDIA driver version is 580.x.x (13.0). This GPU requires a minimum of 20 CPU and 80 GiB of memory.
- Fractional GPU with the current NVIDIA driver version is 580.x.x
(13.0), type: GRID. Fractional
GPU (
0.5GPU, 48 GB VRAM) requires 10, 12, 14, or 16 CPU and at least 40 GiB of memory (up to 4 GiB per CPU).
L4 GPU with the current NVIDIA driver version: 580.x.x (13.0). For L4 GPU, you must use a minimum of 4 CPU and 16 GiB of memory. Fractional GPU is not supported on L4 GPU.
Fractional GPU
Fractional GPU lets you to reduce your GPU allocation for models that don't require a full GPU, while still benefiting from the GPU's advanced architecture and performance. With a fractional GPU, you can have a more cost effective solution for inference workloads that don't need the full capabilities of a full-sized NVIDIA RTX PRO 6000 Blackwell GPU. See Known issues for fractional GPU limitations.
Supported regions
The following regions are supported by the NVIDIA RTX PRO 6000 Blackwell GPU:
asia-southeast1(Singapore).asia-south2(Delhi, India).europe-west4(Netherlands)Low CO2
us-central1(Iowa)Low CO2
Fractional NVIDIA RTX PRO 6000 Blackwell GPU (Preview) are available in the following regions:
The following regions are supported by the L4 GPU:
asia-southeast1(Singapore)asia-south1(Mumbai) . This region is available by invitation only. Contact your Google Account team if you are interested in this region.europe-west1(Belgium)Low CO2
europe-west4(Netherlands)Low CO2
us-central1(Iowa)Low CO2 . Additional resource scaling in this region might require a quota increase request. Contact your Google Account team if you are interested in this region.
us-east4(Northern Virginia) . Additional resource scaling in this region might require a quota increase request. Contact your Google Account team if you are interested in this region.
Pricing impact
See Cloud Run pricing for GPU pricing details. Note the following requirements and considerations:
- There are no per-request fees. You must use instance-based billing to use the GPU feature, minimum instances are charged at the full rate even when idle.
- There is a difference in cost between GPU zonal redundancy and non-zonal redundancy. See Cloud Run pricing for GPU pricing details.
- When you deploy a GPU-enabled Cloud Run service or function from source
code, Cloud Run builds your source code using the
e2-highcpu-8machine type instead of the defaulte2-standard-2machine type. The larger machine type provides higher CPU support, and higher network bandwidth, which results in faster build times. - The CPU and memory configurations of your resource impact pricing.
- GPU is billed for the entire duration of the instance lifecycle.
GPU zonal redundancy options
By default, Cloud Run deploys your service across multiple zones within a region. This architecture provides inherent resilience: if a zone experiences an outage, Cloud Run automatically routes traffic away from the affected zone to healthy zones within the same region.
When working with GPU resources, keep in mind GPU resources have specific capacity constraints. During a zonal outage, the standard failover mechanism for GPU workloads relies on sufficient unused GPU capacity being available in the remaining healthy zones. Due to the constrained nature of GPUs, this capacity might not always be available.
To increase the availability of your GPU-accelerated services during zonal outages, you can configure zonal redundancy specifically for GPU:
Zonal Redundancy Turned On (default): Cloud Run reserves GPU capacity for your service across multiple zones. This significantly increases the probability that your service can successfully handle traffic rerouted from an affected zone, offering higher reliability during zonal failures with additional cost per GPU second.
Zonal Redundancy Turned Off: Cloud Run attempts failover for GPU workloads on a best-effort basis. Traffic is routed to other zones only if sufficient GPU capacity is available at that moment. This option does not guarantee reserved capacity for failover scenarios but results in a lower cost per GPU second.
SLA
The SLA for Cloud Run GPU depends on whether the service uses the zonal redundancy or non-zonal redundancy option. Refer to the SLA page for details.
Request a quota increase
Quota for Cloud Run nvidia-rtx-pro-6000 GPU is granted in milliGPUs.
A full (1) nvidia-rtx-pro-6000 GPU consumes 1,000 milliGPUs of quota, and a
fractional (0.5) nvidia-rtx-pro-6000 GPU consumes 500 milliGPUs of quota.
Projects using nvidia-rtx-pro-6000 GPU in a region for the first time will be
automatically granted 3,000 milliGPU quota (zonal redundancy off) when the first
deployment is created. This is equivalent to 3 full GPUs or 6 fractional (0.5)
GPUs. Projects using Cloud Run nvidia-l4 GPU in a region for the
first time are automatically granted 3 GPU quota (zonal redundancy off) when the
first deployment is created.
If you need additional Cloud Run GPUs, you must request a quota increase for your Cloud Run service. Use the links provided in the following buttons to request the quota you need.
| Quota needed | Quota link |
|---|---|
| NVIDIA RTX PRO 6000 Blackwell GPU with zonal redundancy turned off (lower price) | Request GPU quota without zonal redundancy |
| NVIDIA RTX PRO 6000 Blackwell GPU with zonal redundancy turned on (higher price) | Request GPU quota with zonal redundancy |
| L4 GPU with zonal redundancy turned off (lower price) | Request GPU quota without zonal redundancy |
| L4 GPU with zonal redundancy turned on (higher price) | Request GPU quota with zonal redundancy |
For more information on requesting quota increases, see How to increase quota.
Before you begin
The following list describes requirements and limitations when using GPU in Cloud Run:
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
-
Create a project: To create a project, you need the Project Creator role
(
roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.createpermission. Learn how to grant roles.
-
Verify that billing is enabled for your Google Cloud project.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
-
Create a project: To create a project, you need the Project Creator role
(
roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.createpermission. Learn how to grant roles.
-
Verify that billing is enabled for your Google Cloud project.
Enable the Cloud Run API, if it is not already enabled.
Roles required to enable APIs
To enable APIs, you need the
serviceusage.services.enablepermission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.- Request required quota.
- Consult Best practices: AI inference on Cloud Run with GPU for recommendations on building your container image and loading large models.
- Make sure your Cloud Run service has the following configurations:
- Configure the billing settings to instance-based billing. Services that are set to instance-based billing can still scale to zero.
- The minimum memory and CPU requirements based on the type of GPU you are using.
- An optimal maximum concurrency for your GPU usage.
- Maximum instances must be set to a number that is below the allowed quota per project per region for GPU. See About GPU and maximum instances.
Required roles
To get the permissions that you need to configure and deploy Cloud Run services, ask your administrator to grant you the following IAM roles on services:
- Cloud Run Developer (
roles/run.developer) - the Cloud Run service - Service Account User (
roles/iam.serviceAccountUser) - the service identity
If you are deploying a service or function from source code, you must also have additional roles granted to you on your project and Cloud Build service account.
For a list of IAM roles and permissions that are associated with Cloud Run, see Cloud Run IAM roles and Cloud Run IAM permissions. If your Cloud Run service interfaces with Google Cloud APIs, such as Cloud Client Libraries, see the service identity configuration guide. For more information about granting roles, see deployment permissions and manage access.
Configure a Cloud Run service with GPU
Any configuration change leads to the creation of a new revision. Subsequent revisions will also automatically get this configuration setting unless you make explicit updates to change it.
You can use the Google Cloud console, Google Cloud CLI or YAML to configure GPU.
Console
In the Google Cloud console, go to the Cloud Run page:
Select Services from the Cloud Run navigation menu:
If you are configuring a new service, click Deploy container and fill out the initial service settings page. Then click Containers, Networking, Security to expand the service configuration page.
If you are configuring an existing service, click the service.
Click the Containers tab.
Configure CPU, memory, concurrency, execution environment, and startup probe following the recommendations in Before you begin.
Select GPU.
Select the GPU type and Number of GPUs from the respective menus.
By default, zonal redundancy is turned on for new services. Selecting GPU shows the GPU redundancy options.
- To turn off zonal redundancy, select No zonal redundancy.
- To turn on zonal redundancy, select Zonal redundancy.
Click Create for a new service. Click View diff & redeploy, then Deploy changes for an existing service.
gcloud
To create a service with GPU enabled, use the
gcloud run deploy command:
To deploy a container:
gcloud run deploy SERVICE \ --image IMAGE_URL \ --gpu GPU_NUMBER \ --gpu-type GPU_TYPE
Replace the following:
- SERVICE: the name of your Cloud Run service.
IMAGE_URL: a reference to the container image, for example,us-docker.pkg.dev/cloudrun/container/hello:latest. If you use Artifact Registry, the repository REPO_NAME must already be created. The URL follows the format ofLOCATION-docker.pkg.dev/PROJECT_ID/REPO_NAME/PATH:TAG.- GPU_NUMBER: the number of GPU to attach (
1for a full GPU, or0.5for a fractional GPU). - GPU_TYPE: the GPU type (
nvidia-rtx-pro-6000ornvidia-l4).
To deploy from source code:
gcloud run deploy SERVICE \ --source . \ --gpu GPU_NUMBER \ --gpu-type GPU_TYPE
To update the GPU configuration for a service, use the
gcloud run services update
command. For example, to update an existing service that specifies a container
image:
gcloud run services update SERVICE \ --image IMAGE_URL \ --cpu CPU \ --memory MEMORY \ --no-cpu-throttling \ --gpu GPU_NUMBER \ --gpu-type GPU_TYPE \ --max-instances MAX_INSTANCE \ --GPU_ZONAL_REDUNDANCY
Replace the following:
- SERVICE: the name of your Cloud Run service.
IMAGE_URL: a reference to the container image, for example,us-docker.pkg.dev/cloudrun/container/hello:latest. If you use Artifact Registry, the repository REPO_NAME must already be created. The URL follows the format ofLOCATION-docker.pkg.dev/PROJECT_ID/REPO_NAME/PATH:TAG.- CPU: the number of CPUs. Ensure this meets the minimum requirements.
- MEMORY: the amount of memory. Ensure this meets the minimum requirements.
- GPU_NUMBER: the number of GPU to attach (
1for a full GPU, or0.5for a fractional GPU). - GPU_TYPE: the GPU type. For NVIDIA RTX PRO 6000 Blackwell GPU,
enter
nvidia-rtx-pro-6000. For L4 GPU, enter valuenvidia-l4(nvidia-L4 lowercase L, not numeric value fourteen). - MAX_INSTANCE: the maximum number of instances. This number can't exceed the GPU quota allocated for your project.
- GPU_ZONAL_REDUNDANCY:
no-gpu-zonal-redundancyto turn off zonal redundancy, orgpu-zonal-redundancyto turn on zonal redundancy.
YAML
If you are creating a new service, skip this step. If you are updating an existing service, download its YAML configuration:
gcloud run services describe SERVICE --format export > service.yaml
Update the
nvidia.com/gpu:attribute andnodeSelector::
run.googleapis.com/accelerator:apiVersion: serving.knative.dev/v1 kind: Service metadata: name: SERVICE spec: template: metadata: annotations: autoscaling.knative.dev/maxScale: 'MAX_INSTANCE' run.googleapis.com/cpu-throttling: 'false' run.googleapis.com/gpu-zonal-redundancy-disabled: 'GPU_ZONAL_REDUNDANCY' spec: containers: - image: IMAGE_URL ports: - containerPort: CONTAINER_PORT name: http1 resources: limits: cpu: 'CPU' memory: 'MEMORY' nvidia.com/gpu: 'GPU_NUMBER' # Optional: use a longer startup probe to allow long starting containers startupProbe: failureThreshold: 1800 periodSeconds: 1 tcpSocket: port: CONTAINER_PORT timeoutSeconds: 1 nodeSelector: run.googleapis.com/accelerator: GPU_TYPE
Replace the following:
- SERVICE: the name of your Cloud Run service.
IMAGE_URL: a reference to the container image, for example,us-docker.pkg.dev/cloudrun/container/hello:latest. If you use Artifact Registry, the repository REPO_NAME must already be created. The URL follows the format ofLOCATION-docker.pkg.dev/PROJECT_ID/REPO_NAME/PATH:TAG.- CONTAINER_PORT: the container port set for your service.
- CPU: the number of CPUs. Ensure this meets the minimum requirements.
- MEMORY: the amount of memory. Ensure this meets the minimum requirements.
- GPU_NUMBER: the number of GPU to attach (
1for a full GPU, or0.5for a fractional GPU). - GPU_TYPE: the GPU type. For NVIDIA RTX PRO 6000 Blackwell GPU,
enter
nvidia-rtx-pro-6000. For L4 GPU, enter valuenvidia-l4(nvidia-L4 lowercase L, not numeric value fourteen). - MAX_INSTANCE: the maximum number of instances. This number can't exceed the GPU quota allocated for your project.
- GPU_ZONAL_REDUNDANCY:
falseto turn on GPU zonal redundancy, ortrueto turn it off.
Create or update the service using the following command:
gcloud run services replace service.yaml
The
gcloud run services replacecommand defaults to usingservice.yamlfile if present.
Terraform
To learn how to apply or remove a Terraform configuration, see Basic Terraform commands.
Add the following to agoogle_cloud_run_v2_service
resource in your Terraform configuration:resource "google_cloud_run_v2_service" "default" {
provider = google-beta
name = "SERVICE"
location = "europe-west1"
template {
gpu_zonal_redundancy_disabled = "GPU_ZONAL_REDUNDANCY"
containers {
image = "IMAGE_URL"
resources {
limits = {
"cpu" = "CPU"
"memory" = "MEMORY"
"nvidia.com/gpu" = "GPU_NUMBER"
}
}
}
node_selector {
accelerator = "GPU_TYPE"
}
}
}
Replace the following:
- SERVICE: the name of your Cloud Run service.
- GPU_ZONAL_REDUNDANCY:
falseto turn on GPU zonal redundancy, ortrueto turn it off. IMAGE_URL: a reference to the container image, for example,us-docker.pkg.dev/cloudrun/container/hello:latest. If you use Artifact Registry, the repository REPO_NAME must already be created. The URL follows the format ofLOCATION-docker.pkg.dev/PROJECT_ID/REPO_NAME/PATH:TAG.- CPU: the number of CPUs. Ensure this meets the minimum requirements.
- MEMORY: the amount of memory. Ensure this meets the minimum requirements.
- GPU_NUMBER: the number of GPU to attach (
1for a full GPU, or0.5for a fractional GPU). - GPU_TYPE: the GPU type. For NVIDIA RTX PRO 6000 Blackwell GPU,
enter
nvidia-rtx-pro-6000. For L4 GPU, enter valuenvidia-l4(nvidia-L4 lowercase L, not numeric value fourteen).
View GPU settings
To view the current GPU settings for your Cloud Run service:
Console
In the Google Cloud console, go to the Cloud Run Services page:
Click the service you are interested in to open the Service details page.
Click the Containers tab to view the settings.
gcloud
Use the following command:
gcloud run services describe SERVICE
Locate the GPU setting in the returned configuration.
Remove GPU
You can remove GPU using the Google Cloud console, the Google Cloud CLI, or YAML.
Console
In the Google Cloud console, go to the Cloud Run page:
Select Services from the Cloud Run navigation menu:
If you are configuring a new service, click Deploy container and fill out the initial service settings page. Then click Containers, Networking, Security to expand the service configuration page.
If you are configuring an existing service, click the service.
Click the Containers tab.
Clear GPU.
Click Create for a new service. Click View diff & redeploy, then Deploy changes for an existing service.
gcloud
To remove GPU, set the number of GPU to 0 using the
gcloud run services update
command:
gcloud run services update SERVICE --gpu 0
Replace SERVICE with the name of your Cloud Run service.
YAML
If you are creating a new service, skip this step. If you are updating an existing service, download its YAML configuration:
gcloud run services describe SERVICE --format export > service.yaml
Delete the
nvidia.com/gpu:and thenodeSelector: run.googleapis.com/accelerator: GPU_TYPElines.Create or update the service using the following command:
gcloud run services replace service.yaml
The
gcloud run services replacecommand defaults to usingservice.yamlfile if present.
Driver libraries
By default, all of the NVIDIA RTX PRO 6000 Blackwell GPU and NVIDIA L4 GPU driver libraries
are mounted under /usr/local/nvidia/lib64. Cloud Run automatically
appends this path to the LD_LIBRARY_PATH environment variable
(i.e. ${LD_LIBRARY_PATH}:/usr/local/nvidia/lib64) of the container with the
GPU. This allows the dynamic linker to find the NVIDIA driver libraries. The
linker searches and resolves paths in the order you list in the LD_LIBRARY_PATH
environment variable. Any values you specify in this variable take precedence
over the default Cloud Run driver libraries path /usr/local/nvidia/lib64.
If you want to use a CUDA version greater than 13.0,
the easiest way is to depend on a newer NVIDIA base image
with forward compatibility packages already installed. Another option is to
manually install the NVIDIA forward compatibility packages
and add them to LD_LIBRARY_PATH. Consult NVIDIA's compatibility matrix
to determine which CUDA versions are forward compatible with the provided NVIDIA
driver version.
About GPU and maximum instances
The number of instances with GPU is limited in two ways:
- The Maximum instances setting limits
the number of instances per service. This can't be set higher than the
total GPU quota per project per region divided by the GPU allocation per instance. For example, if your project has 3,000 milliGPUs of
nvidia-rtx-pro-6000quota and your service uses--gpu 0.5(500 milliGPUs per instance), you can set maximum instances up to6. - The quota of GPUs allowed per project per region. This limits the number of instances across services in the same region.
What's next
See Run AI inference on Cloud Run with GPU for tutorials.