Managed Service for Apache Spark has API quota limits that are enforced at the project and
region level. The quotas reset every 60 seconds (1 minute).
**Handling Managed Service for Apache Spark Quota Errors** : If you exceed a Managed Service for Apache Spark quota limit, a [RESOURCE_EXHAUSTED](https://docs.cloud.google.com/apis/design/errors#handling_errors) (HTTP code 429) is generated, and the corresponding Managed Service for Apache Spark API request fails. However, since your project's Managed Service for Apache Spark quota is refreshed every 60 seconds, you can retry your request after 1 minute has elapsed following the failure.

The following table lists the specific and default per-project
Managed Service for Apache Spark API quota types, quota limits, and the methods to
which they apply.

| Quota Type | Limit | API Methods or Description |
|---|---|---|
| ClusterOperationRequestsPerMinutePerProjectPerRegion | 200 | CancelOperation (cancelling batch operation quota is shared with cancelling cluster operation quota). |
| BatchOperationRequestsPerMinutePerProjectPerRegion | 200 | CreateBatch, DeleteBatch |
| SessionOperationRequestsPerMinutePerProjectPerRegion | 200 | CreateSession, DeleteSession, TerminateSession |
| DefaultRequestsPerMinutePerProjectPerRegion | 7,500 | GetBatch, ListBatches, GetSession, ListSessions |
| ActiveOperationsPerProjectPerRegion | 5,000 | Limit on the total number of concurrent active operations of all types in a project in a region. |

> [!IMPORTANT]
> **Increasing Resource Quota Limits** : Open the Google Cloud [Identity and Access Management (IAM) \& Admin Quotas](https://console.cloud.google.com/iam-admin/quotas) page, and select the resources you want to modify. Then click **Edit Quotas** at the top of the page to start the quota increase process.

## Managed Service for Apache Spark serverless DCU quota

Starting with Managed Service for Apache Spark serverless runtime version `4.0`, a
project's Data Compute Unit (DCU) quota
(`dataproc.googleapis.com/data_compute_units`), instead of
Compute Engine quota, manages and limits
Managed Service for Apache Spark serverless batch workload and interactive session
compute resource usage (underlying Compute Engine quota usage is
redirected to a dedicated Managed Service for Apache Spark producer project).

You can view and adjust your DCU quota on the Google Cloud console
[**Quotas** page](https://console.cloud.google.com/iam-admin/quotas)
in Dataproc API. You can enable the
[quota adjuster](https://docs.cloud.google.com/docs/quotas/quota-adjuster.md.txt) to increase your DCU
quota when your usage approaches its limit.

> [!NOTE]
> **Note:** When onboarding to Managed Service for Apache Spark serverless runtime version \`4.0\`, please make sure you have sufficient DCU quota for your expected workload size and number of concurrent sessions/batches.

### Calculate DCU consumption

DCU usage is calculated **per node** from the VM shape, using the same formula
as Managed Service for Apache Spark serverless [DCU
pricing](https://cloud.google.com/dataproc-serverless/pricing):

- **0.6 DCU** per vCPU.
- **0.1 DCU** per GB of memory, up to 8 GB per vCPU.
- **0.2 DCU** per GB of memory above 8 GB per vCPU (extended memory).
- **Premium tier** nodes count **1.5x** the DCUs of the same shape on standard tier.

The DCUs of a node pool are rounded **up** to the nearest integer:

    DCUs = ceiling(nodes × DCUs per node)

**Example** : A standard tier node with 4 vCPUs and 16 GB of memory uses
`4 × 0.6 + 16 × 0.1 = 4` DCUs. A batch with 1 driver and 10 executors of this
shape uses 44 DCUs. The same batch on premium tier uses 66 DCUs.

### Limitations

The following limitations apply to DCU quotas:

- **Runtime versions** : DCU quotas apply to Managed Service for Apache Spark serverless runtime version `4.0` and later. Workloads that use runtime versions prior to `4.0` don't consume DCU quota and continue to consume a project's Compute Engine quota.
- **Unsupported quotas**: GPU, reservation, and global Compute Engine quotas are not supported in the DCU quota model.

## Other Google Cloud quotas

Managed Service for Apache Spark batches use other Google Cloud products.
These products have project-level quotas, which include quotas that apply to
Managed Service for Apache Spark use. Some services are
[required](https://docs.cloud.google.com/managed-spark/quotas-serverless.md.txt#required-services) to use Managed Service for Apache Spark, such as [Compute Engine](https://docs.cloud.google.com/compute "Compute Engine") and [Cloud Storage](https://docs.cloud.google.com/storage "gcs_name"). Other services, such as [BigQuery](https://docs.cloud.google.com/bigquery "bigquery_name") and [Bigtable](https://docs.cloud.google.com/bigtable "Bigtable"), can
[optionally](https://docs.cloud.google.com/managed-spark/quotas-serverless.md.txt#optional-services) be used with Managed Service for Apache Spark.

### Required services

The following services, which enforce quota limits, are required to create
Managed Service for Apache Spark batches.

#### Compute Engine

> [!NOTE]
> **Note:** The Compute Engine quotas mentioned in this section are not required for runtime version \`4.0\` and later. For more information, see [DCU quota](https://docs.cloud.google.com/managed-spark/quotas-serverless.md.txt#dataproc-serverless-dcu-quota).

Managed Service for Apache Spark batch workloads can select a variety of
Compute Engine resources:

- To mitigate obtainability issues with CPUs and disks,
  Managed Service for Apache Spark supports fallback across multiple machine
  families and disk types. For example, in the Premium tier, a batch workload
  can fall back to `N2` with persistent SSD, `N2D` with local SSD, `N2D` with
  persistent disk, `N4` with hyperdisk balanced, or `N4D` with hyperdisk
  balanced.

- A batch workload fails if it has insufficient quota to consume the fallback
  selection resources.

The following table lists [Compute Engine resource
quotas](https://docs.cloud.google.com/compute/resource-usage) that apply to Managed Service for Apache Spark
batch workloads.

| Compute Tier | Quota |
|---|---|
| Standard | `CPUS` |
| Premium | - `N2_CPUS` - `N2D_CPUS` - `CPUS_PER_VM_FAMILY` with N4 VM Family - `CPUS_PER_VM_FAMILY` with N4D VM Family |
| Standard | `DISKS_TOTAL_GB` |
| Premium | - `DISKS_TOTAL_GB` - `LOCAL_SSD_TOTAL_GB_PER_VM_FAMILY` with N2 and N2D VM Families - `SSD-TOTAL-GB` or `Persistent Disk SSD (GB)` - `HDB-TOTAL-GB` or `Hyperdisk Balanced Capacity (GB)` |
| L4 | `NVIDIA_L4_GPUS` |
| A100 40GB | `NVIDIA_A100_GPUS` |
| A100 80GB | `NVIDIA_A100_80GB_GPUS` |

The [Compute Engine quotas](https://docs.cloud.google.com/compute/resource-usage.md.txt " Compute Engine Quotas") are split into regional and global limits. These
limits apply to batches you create. For example, to run a Spark batch with 4
driver cores (`spark.driver.cores=4`) and two executors with 4 cores each
(`spark.executor.cores=4`), you use 12 virtual CPUs (`4 * 3`). This batch usage
counts against the regional quota limit of 24 virtual CPUs.

> [!IMPORTANT]
> **Free trial quotas:**If you use the Compute Engine free trial, your project will have a limit of 8 CPUs. To increase this limit, you must enable billing on the project.

#### Default batch resources

When you create a batch with default settings, the following
Compute Engine resources are used:

| **Item** | **Resources used** |
|---|---|
| Virtual CPUs | 12 |
| Virtual Machine (VM) Instances | 3 |
| Persistent disk | 1,200 GiB |

#### Logging

Managed Service for Apache Spark saves batch output and logs in
[Cloud Logging](https://docs.cloud.google.com/logging "Cloud Logging"). The [Logging
quota](https://docs.cloud.google.com/logging/quota-policy "Cloud Logging quota") applies to your
Managed Service for Apache Spark batches.

### Optional services

The following services, which have quota limits, can optionally be used with
Managed Service for Apache Spark batches.

#### BigQuery

When reading or writing data into BigQuery, the [BigQuery quota](https://docs.cloud.google.com/bigquery/quota-policy "BigQuery quota") applies.

#### Bigtable

When reading or writing data into Bigtable, the [Bigtable quota](https://docs.cloud.google.com/bigtable/quota "Bigtable quota")
applies.

## Identify workloads with quota or IP address limitations

You can use the following [Logging](https://console.cloud.google.com/logs/query)
queries to identify Managed Service for Apache Spark workloads that reached your
quota or were unable to scale due to IP address exhaustion.

Quota query:

    jsonPayload.@type="type.googleapis.com/google.cloud.dataproc.logging.AutoscalerLog"