Dataproc and Google Cloud Serverless for Apache Spark are now unified under the Managed Service for Apache Spark brand. This change consolidates our managed Spark deployment options into a single umbrella brand that includes the full breadth of our Spark capabilities. No existing functionality is being removed as part of this change, and there will be no impact to the Dataproc API, client library, CLI, or IAM names.

New Dataproc on Compute Engine subminor image versions:

• 2.3.28-debian12, 2.3.28-ml-ubuntu22, 2.3.28-rocky9, 2.3.28-ubuntu22, 2.3.28-ubuntu22-arm

Change

• Dataproc on Compute Engine: Upgraded Apache Zookeeper to version 3.9.5 in image version 2.3.

Fixed

• Upgraded Dataproc Metastore Proxy to v0.0.79 to fix CVEs.
• Fixed CVEs CVE-2026-24308 and CVE-2026-24281.

New Dataproc on Compute Engine subminor image versions:

• 2.1.112-debian11, 2.1.112-rocky8, 2.1.112-ubuntu20, 2.1.112-ubuntu20-arm
• 2.2.80-debian12, 2.2.80-rocky9, 2.2.80-ubuntu22, 2.2.80-ubuntu22-arm
• 2.3.27-debian12, 2.3.27-ml-ubuntu22, 2.3.27-rocky9, 2.3.27-ubuntu22, 2.3.27-ubuntu22-arm

New Serverless for Apache Spark runtime versions:

• 1.2.76
• 2.2.76
• 2.3.29
• 3.0.12

New Dataproc on Compute Engine subminor image versions:

• 2.1.111-debian11, 2.1.111-rocky8, 2.1.111-ubuntu20, 2.1.111-ubuntu20-arm
• 2.2.79-debian12, 2.2.79-rocky9, 2.2.79-ubuntu22, 2.2.79-ubuntu22-arm
• 2.3.26-debian12, 2.3.26-ml-ubuntu22, 2.3.26-rocky9, 2.3.26-ubuntu22, 2.3.26-ubuntu22-arm

Fixed

• Fixed CVEs CVE-2025-58057, CVE-2025-53864, CVE-2025-68161, CVE-2025-48924, and CVE-2025-33042.
• Upgraded Dataproc Metastore Proxy to v0.0.78 to fix CVEs.
• Default JDK is set to Temurin JDK - 11.0.30 in all 2.1, 2.2 and 2.3 images.

New Serverless for Apache Spark runtime versions:

• 1.2.75
• 2.2.75
• 2.3.28
• 3.0.11

Dataproc on Compute Engine: The following subminor image versions announced on March 08, 2026 have been rolled back:

• 2.1.110-debian11, 2.1.110-rocky8, 2.1.110-ubuntu20, 2.1.110-ubuntu20-arm
• 2.2.78-debian12, 2.2.78-rocky9, 2.2.78-ubuntu22, 2.2.78-ubuntu22-arm
• 2.3.25-debian12, 2.3.25-ml-ubuntu22, 2.3.25-rocky9, 2.3.25-ubuntu22, 2.3.25-ubuntu22-arm

New Dataproc on Compute Engine subminor image versions:

• 2.1.110-debian11, 2.1.110-rocky8, 2.1.110-ubuntu20, 2.1.110-ubuntu20-arm
• 2.2.78-debian12, 2.2.78-rocky9, 2.2.78-ubuntu22, 2.2.78-ubuntu22-arm
• 2.3.25-debian12, 2.3.25-ml-ubuntu22, 2.3.25-rocky9, 2.3.25-ubuntu22, 2.3.25-ubuntu22-arm

Rollback Notice: These image versions were rolled back on March 11, 2026.

Fixed

• Fixed CVEs CVE-2025-58057, CVE-2025-53864, CVE-2025-68161, CVE-2025-48924 (partial), and CVE-2025-33042.
• Upgraded Dataproc Metastore Proxy to v0.0.78 to fix CVEs.

New Serverless for Apache Spark runtime versions:

• 1.2.74
• 2.2.74
• 2.3.27
• 3.0.10

Announcement

Serverless for Apache Spark 3.0 runtime version:

• Updated Spark version to 4.0.1.
• Updated BigQuery Spark Connector version to 0.44.0.
• Applied patch for SPARK-52601.
• Applied patch for SPARK-52614.

Announcement

Serverless for Apache Spark 1.2, 2.2 and 2.3 runtime versions:

• Updated Delta Lake version to 3.2.1.

New Serverless for Apache Spark runtime versions:

• 1.2.73
• 2.2.73
• 2.3.26

New Serverless for Apache Spark runtime versions:

• 1.2.72
• 2.2.72
• 2.3.25
• 3.0.9

New Dataproc on Compute Engine subminor image versions:

• 2.0.160-debian10, 2.0.160-rocky8, 2.0.160-ubuntu18
• 2.1.109-debian11, 2.1.109-rocky8, 2.1.109-ubuntu20, 2.1.109-ubuntu20-arm
• 2.2.77-debian12, 2.2.77-rocky9, 2.2.77-ubuntu22, 2.2.77-ubuntu22-arm
• 2.3.24-debian12, 2.3.24-ml-ubuntu22, 2.3.24-rocky9, 2.3.24-ubuntu22, 2.3.24-ubuntu22-arm

Fixed

Fixed an issue that could cause gsutil failures on clusters using custom images built from new base images. This problem occurred when the gcloud core/universe_domain property was not set in the build environment, resulting in a malformed /etc/boto.cfg. The image build process now ensures a correct default configuration, restoring proper Google Cloud Storage connectivity.

New Serverless for Apache Spark runtime versions:

• 1.2.71
• 2.2.71
• 2.3.24
• 3.0.8

Hyperdisks can now be added to a Dataproc cluster.

New Dataproc on Compute Engine subminor image versions:

• 2.0.159-debian10, 2.0.159-rocky8, 2.0.159-ubuntu18
• 2.1.108-debian11, 2.1.108-rocky8, 2.1.108-ubuntu20, 2.1.108-ubuntu20-arm
• 2.2.76-debian12, 2.2.76-rocky9, 2.2.76-ubuntu22, 2.2.76-ubuntu22-arm
• 2.3.23-debian12, 2.3.23-ml-ubuntu22, 2.3.23-rocky9, 2.3.23-ubuntu22, 2.3.23-ubuntu22-arm

New Serverless for Apache Spark runtime versions:

• 1.2.70
• 2.2.70
• 2.3.23
• 3.0.7

New Serverless for Apache Spark runtime versions:

• 1.2.69
• 2.2.69
• 2.3.22
• 3.0.5

Feature

Serverless for Apache Spark: Added support for removing conscrypt from Serverless for Apache Spark 2.3+ runtimes using the dataproc.artifacts.remove property .

Feature

Dataproc on Compute Engine: Sharing snapshot diagnostic data: Setting the --tarball-access=GOOGLE_DATAPROC_DIAGNOSE flag with the gcloud dataproc clusters diagnose command shares all of the output Cloud Storage bucket contents with Google Cloud support if uniform bucket-level access is enabled on the output Cloud Storage bucket. If object-level access control is enabled on the output Cloud Storage bucket, only the generated diagnostic tar file is shared.

New Dataproc on Compute Engine subminor image versions:

• 2.0.158-debian10, 2.0.158-ubuntu18, 2.0.158-rocky8
• 2.1.107-debian11, 2.1.107-ubuntu20, 2.1.107-ubuntu20-arm, 2.1.107-rocky8
• 2.2.75-debian12, 2.2.75-ubuntu22, 2.2.75-ubuntu22-arm, 2.2.75-rocky9
• 2.3.22-debian12, 2.3.22-ml-ubuntu22, 2.3.22-rocky9, 2.3.22-ubuntu22, 2.3.22-ubuntu22-arm

Feature

Zookeeper version upgraded to 3.9.4 in Dataproc image 2.3.22.

Feature

Parquet CLI version upgraded to 1.15.2 in 2.1 and 2.2 images.

Feature

Delta subminor version upgraded to 3.2.1 in Dataproc on Compute Engine image 2.2 and 2.3.

Feature

Apache Pig is now available in ARM images.

Feature

Added a new dataproc:pypi.repository property to customize the PyPI repository used for pip. The value can be a URL, or google to use a Google-hosted cache of PyPI, accessible without public internet connectivity. Starting in image version 3.1, google will be the default; to opt out and return to public PyPI, use the value pypi.

Change

Removed use of deprecated Hadoop configuration properties fs.default.name and yarn.resourcemanager.system-metrics-publisher.enabled.

Fixed

Fixed a bug in the ARM image that prevented connecting to a Dataproc Metastore instance with a gRPC protocol endpoint.

Fixed

Fixed the spark.driver.extraClassPath delimiter for the Jupyter SparkMonitor Listener.

Upcoming Spark data lineage changes See the upcoming May, 2026 Dataproc and Serverless for Apache Spark release notes for an announcement of a change that will automatically enable Dataproc Spark data lineage and Serverless for Apache Spark data lineage when you enable the Data Lineage API (see Control lineage ingestion for a service) without requiring additional project, cluster, batch workload, or interactive session settings.

New Serverless for Apache Spark runtime versions:

• 3.0.4

Announcement

Dataproc on Compute Engine: The following subminor image versions announced on January 24, 2026 have been rolled back:

• 2.0.157-debian10, 2.0.157-ubuntu18, 2.0.157-rocky8
• 2.1.106-debian11, 2.1.106-ubuntu20, 2.1.106-ubuntu20-arm, 2.1.106-rocky8
• 2.2.74-debian12, 2.2.74-ubuntu22, 2.2.74-ubuntu22-arm, 2.2.74-rocky9
• 2.3.21-debian12, 2.3.21-ml-ubuntu22, 2.3.21-rocky9, 2.3.21-ubuntu22, 2.3.21-ubuntu22-arm

New Dataproc on Compute Engine subminor image versions:

• 2.0.157-debian10, 2.0.157-ubuntu18, 2.0.157-rocky8
• 2.1.106-debian11, 2.1.106-ubuntu20, 2.1.106-ubuntu20-arm, 2.1.106-rocky8
• 2.2.74-debian12, 2.2.74-ubuntu22, 2.2.74-ubuntu22-arm, 2.2.74-rocky9
• 2.3.21-debian12, 2.3.21-ml-ubuntu22, 2.3.21-rocky9, 2.3.21-ubuntu22, 2.3.21-ubuntu22-arm

Rollback Notice: These image versions were rolled back on January 30, 2026.

Feature

Upgraded the Delta subminor version to 3.2.1 in images 2.2 and 2.3.

Feature

Apache Pig is now available in ARM images.

Feature

Added a new dataproc:pypi.repository property to customize the PyPI repository that pip uses. The value can be a internet-accessible URL, or you can specify google to use a Google-hosted cache of PyPI, which is accessible without public internet connectivity. Starting with Managed Service for Apache Spark image version 3.1, google will be the default (you can specify the pypi property value to use public PyPI instead of the default google 3.1 value).

Change

Removed the use of deprecated Hadoop configuration properties fs.default.name and yarn.resourcemanager.system-metrics-publisher.enabled.

Fixed

Fixed a bug in the ARM image that prevented connecting to a Dataproc Metastore instance with a gRPC protocol endpoint.

Fixed

Fixed the spark.driver.extraClassPath delimiter for the Jupyter SparkMonitor Listener.

New Serverless for Apache Spark runtime versions:

• 1.2.67
• 2.2.67
• 2.3.20
• 3.0.3

Fixed

• Applied patch for SPARK-48292 in Serverless for Apache Spark 1.2 and 2.2 runtimes.

Dataproc on Compute Engine: The following subminor image versions announced on January 06, 2026 have been rolled back:

• 2.0.156-debian10, 2.0.156-ubuntu18, 2.0.156-rocky8
• 2.1.105-debian11, 2.1.105-ubuntu20, 2.1.105-ubuntu20-arm, 2.1.105-rocky8
• 2.2.73-debian12, 2.2.73-ubuntu22, 2.2.73-ubuntu22-arm, 2.2.73-rocky9
• 2.3.20-debian12, 2.3.20-ml-ubuntu22, 2.3.20-rocky9, 2.3.20-ubuntu22, 2.3.20-ubuntu22-arm

New Serverless for Apache Spark runtime versions:

• 1.2.66
• 2.2.66
• 2.3.19
• 3.0.1

New Dataproc on Compute Engine subminor image versions:

• 2.0.156-debian10, 2.0.156-ubuntu18, 2.0.156-rocky8
• 2.1.105-debian11, 2.1.105-ubuntu20, 2.1.105-ubuntu20-arm, 2.1.105-rocky8
• 2.2.73-debian12, 2.2.73-ubuntu22, 2.2.73-ubuntu22-arm, 2.2.73-rocky9
• 2.3.20-debian12, 2.3.20-ml-ubuntu22, 2.3.20-rocky9, 2.3.20-ubuntu22, 2.3.20-ubuntu22-arm

Rollback Notice: These image versions were rolled back on January 13, 2026.

Feature

Added a new property dataproc:pypi.repository to customize the PyPI repository used for pip. The value can be a URL, or google to use a Google-hosted cache of PyPI, accessible without public internet connectivity. Starting in image version 3.1, google will be the default; to opt out and return to public PyPI, use the value pypi.

Change

Removed use of deprecated Hadoop configuration properties fs.default.name and yarn.resourcemanager.system-metrics-publisher.enabled.

Fixed

Fixed the spark.driver.extraClassPath delimiter for the Jupyter SparkMonitor Listener.

New Serverless for Apache Spark runtime versions:

• 1.2.65
• 2.2.65
• 2.3.18

The following Dataproc on Compute Engine subminor image versions, released on December 05, 2025, are now blocklisted:

• 2.2.72-debian12, 2.2.72-ubuntu22, 2.2.72-ubuntu22-arm, 2.2.72-rocky9
• 2.3.19-debian12, 2.3.19-ubuntu22, 2.3.19-ubuntu22-arm, 2.3.19-ml-ubuntu22, 2.3.19-rocky9

New Dataproc on Compute Engine subminor image versions:

• 2.0.155-debian10, 2.0.155-ubuntu18, 2.0.155-rocky8
• 2.1.104-debian11, 2.1.104-ubuntu20, 2.1.104-ubuntu20-arm, 2.1.104-rocky8
• 2.2.72-debian12, 2.2.72-ubuntu22, 2.2.72-ubuntu22-arm, 2.2.72-rocky9
• 2.3.19-debian12, 2.3.19-ubuntu22, 2.3.19-ubuntu22-arm, 2.3.19-ml-ubuntu22, 2.3.19-rocky9

Feature

dataproc-ml is available by default in 2.3 -ml images.

Serverless for Apache Spark: Runtime version 3.0 is now generally available. This version simplifies onboarding, improves reliability, reduces startup latency, and adds support for Spark 4.0.

• Features and improvements:

  • Regional and multi-zonal workloads are used by default to increase obtainability of compute resources
  • Faster startup than previous runtimes
  • Fast resource cleanup that allows faster release of VPC IPs after workload completion
  • End-user credentials are used for all workloads by default
  • New bigquery Spark catalog, pre-configured for out-of-the-box BigQuery native table interactions
  • New Spark Serverless-specific IAM roles
  • New dataproc-rm.googleapis.com API enablement is required

New Dataproc on Compute Engine subminor image versions:

• 2.0.154-debian10, 2.0.154-ubuntu18, 2.0.154-rocky8
• 2.1.103-debian11, 2.1.103-ubuntu20, 2.1.103-ubuntu20-arm, 2.1.103-rocky8
• 2.2.71-debian12, 2.2.71-ubuntu22, 2.2.71-ubuntu22-arm, 2.2.71-rocky9
• 2.3.18-debian12, 2.3.18-ubuntu22, 2.3.18-ubuntu22-arm, 2.3.18-ml-ubuntu22, 2.3.18-rocky9

Change

Dataproc on Compute Engine: Updated Cloud Storage connector:

• Updated to 3.1.10 in image version 2.3.18.
• Updated to 3.0.15 in image version 2.2.71.
• Updated to hadoop3-2.2.30 in image versions 2.1.103 and 2.0.154.