| Component | [3.0.4-debian13/ -ubuntu24/ -ml-ubuntu24/ -rocky9 2026/11/02](https://docs.cloud.google.com/managed-spark/docs/release-notes.md.txt#November_02_2026) | [3.0.3-debian13/ -ubuntu24/ -ml-ubuntu24/ -rocky9 2026/09/30](https://docs.cloud.google.com/managed-spark/docs/release-notes.md.txt#September_30_2026) | [3.0.2-debian13/ -ubuntu24/ -ml-ubuntu24/ -rocky9 2026/09/04](https://docs.cloud.google.com/managed-spark/docs/release-notes.md.txt#September_04_2026) | [3.0.1-debian13/ -ubuntu24/ -ml-ubuntu24/ -rocky9 2026/08/19](https://docs.cloud.google.com/managed-spark/docs/release-notes.md.txt#August_19_2026) | [3.0.0-debian13/ -ubuntu24/ -ml-ubuntu24/ -rocky9 2026/07/15](https://docs.cloud.google.com/managed-spark/docs/release-notes.md.txt#July_15_2026) |
|---|---|---|---|---|---|
| [**Apache Flink** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/flink.md.txt) | [2.2.0](https://nightlies.apache.org/flink/flink-docs-release-2.2/release-notes/flink-2.2/) | [2.2.0](https://nightlies.apache.org/flink/flink-docs-release-2.2/release-notes/flink-2.2/) | [2.2.0](https://nightlies.apache.org/flink/flink-docs-release-2.2/release-notes/flink-2.2/) | [2.2.0](https://nightlies.apache.org/flink/flink-docs-release-2.2/release-notes/flink-2.2/) |   |
| [**Apache Hadoop** ^installed^](https://hadoop.apache.org/) | [3.5.0](https://hadoop.apache.org/docs/r3.5.0/) | [3.5.0](https://hadoop.apache.org/docs/r3.5.0/) | [3.5.0](https://hadoop.apache.org/docs/r3.5.0/) | [3.5.0](https://hadoop.apache.org/docs/r3.5.0/) | [3.5.0](https://hadoop.apache.org/docs/r3.5.0/) |
| [**Apache Hive** ^installed^](https://hive.apache.org/) | [4.2.0](https://archive.apache.org/dist/hive/hive-4.2.0/) | [4.2.0](https://archive.apache.org/dist/hive/hive-4.2.0/) | [4.2.0](https://archive.apache.org/dist/hive/hive-4.2.0/) | [4.2.0](https://archive.apache.org/dist/hive/hive-4.2.0/) | [4.2.0](https://archive.apache.org/dist/hive/hive-4.2.0/) |
| [**Apache Hive WebHCat** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/hivewebhcat.md.txt) | [4.2.0](https://archive.apache.org/dist/hive/hive-4.2.0/) | [4.2.0](https://archive.apache.org/dist/hive/hive-4.2.0/) | [4.2.0](https://archive.apache.org/dist/hive/hive-4.2.0/) | [4.2.0](https://archive.apache.org/dist/hive/hive-4.2.0/) |   |
| [**Apache Iceberg** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/iceberg.md.txt) | [1.11.0](https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.11.0) | [1.11.0](https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.11.0) | [1.11.0](https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.11.0) | [1.11.0](https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.11.0) |   |
| [**Apache Hudi** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/hudi.md.txt) | [1.2.0](https://hudi.apache.org/releases/) | [1.2.0](https://hudi.apache.org/releases/) |   |   |   |
| [**Apache Kafka** ^initialization action^](https://github.com/GoogleCloudDataproc/initialization-actions/tree/master/kafka) | [3.9.2](https://archive.apache.org/dist/kafka/3.9.2/RELEASE_NOTES.html) | [3.9.2](https://archive.apache.org/dist/kafka/3.9.2/RELEASE_NOTES.html) | [3.9.2](https://archive.apache.org/dist/kafka/3.9.2/RELEASE_NOTES.html) | [3.9.2](https://archive.apache.org/dist/kafka/3.9.2/RELEASE_NOTES.html) |   |
| [**Apache Pig** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/pig.md.txt) | [0.18.0](https://pig.apache.org/docs/r0.18.0/) | [0.18.0](https://pig.apache.org/docs/r0.18.0/) | [0.18.0](https://pig.apache.org/docs/r0.18.0/) | [0.18.0](https://pig.apache.org/docs/r0.18.0/) | [0.18.0-SNAPSHOT](https://pig.apache.org/docs/latest/) |
| [**Apache Ranger** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/ranger.md.txt) | [2.8.0](https://ranger.apache.org/index.html) | [2.8.0](https://ranger.apache.org/index.html) | [2.8.0](https://ranger.apache.org/index.html) |   |   |
| [**Apache Spark** ^installed^](https://spark.apache.org/) | [4.1.2](https://spark.apache.org/releases/spark-release-4-1-2.html) | [4.1.2](https://spark.apache.org/releases/spark-release-4-1-2.html) | [4.1.2](https://spark.apache.org/releases/spark-release-4-1-2.html) | [4.1.2](https://spark.apache.org/releases/spark-release-4-1-2.html) | [4.1.1](https://spark.apache.org/releases/spark-release-4-1-1.html) |
| [**Apache Solr** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/solr.md.txt) | [9.10.1](https://solr.apache.org/docs/9_10_1/changes/Changes.html) | [9.10.1](https://solr.apache.org/docs/9_10_1/changes/Changes.html) | [9.10.1](https://solr.apache.org/docs/9_10_1/changes/Changes.html) | [9.10.1](https://solr.apache.org/docs/9_10_1/changes/Changes.html) | [9.7.0](https://solr.apache.org/docs/9_7_0/changes/Changes.html) |
| [**Apache Tez** ^installed^](https://tez.apache.org/) | [0.10.5](https://tez.apache.org/releases/apache-tez-0-10-5.html) | [0.10.5](https://tez.apache.org/releases/apache-tez-0-10-5.html) | [0.10.5](https://tez.apache.org/releases/apache-tez-0-10-5.html) | [0.10.5](https://tez.apache.org/releases/apache-tez-0-10-5.html) | [0.10.5](https://tez.apache.org/releases/apache-tez-0-10-5.html) |
| [**Apache Zeppelin Notebook** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/zeppelin.md.txt) | [0.12.0](https://zeppelin.apache.org/docs/0.12.0/) | [0.12.0](https://zeppelin.apache.org/docs/0.12.0/) | [0.12.0](https://zeppelin.apache.org/docs/0.12.0/) | [0.12.0](https://zeppelin.apache.org/docs/0.12.0/) |   |
| [**Apache Zookeeper** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/zookeeper.md.txt) | [3.9.5](https://zookeeper.apache.org/doc/r3.9.5/releasenotes.html) | [3.9.5](https://zookeeper.apache.org/doc/r3.9.5/releasenotes.html) | [3.9.5](https://zookeeper.apache.org/doc/r3.9.5/releasenotes.html) | [3.9.5](https://zookeeper.apache.org/doc/r3.9.5/releasenotes.html) | [3.9.3](https://zookeeper.apache.org/doc/r3.9.3/releasenotes.html) |
| [**BigQuery Connector** ^installed^](https://docs.cloud.google.com/managed-spark/docs/concepts/connectors/bigquery.md.txt) | [0.44.1-Preview](https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases/tag/0.44.1) | [0.44.1-Preview](https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases/tag/0.44.1) | [0.44.1-Preview](https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases/tag/0.44.1) | [0.44.1-Preview](https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases/tag/0.44.1) | [0.44.1-Preview](https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases/tag/0.44.1) |
| [**Cloud Storage Connector** ^installed^](https://docs.cloud.google.com/managed-spark/docs/concepts/connectors/cloud-storage.md.txt#clusters) | [4.0.4](https://github.com/GoogleCloudDataproc/hadoop-connectors/releases/tag/v4.0.4) | [4.0.4](https://github.com/GoogleCloudDataproc/hadoop-connectors/releases/tag/v4.0.4) | [3.1.13](https://github.com/GoogleCloudDataproc/hadoop-connectors/releases/tag/v3.1.13) | [3.1.13](https://github.com/GoogleCloudDataproc/hadoop-connectors/releases/tag/v3.1.13) | [3.1.13](https://github.com/GoogleCloudDataproc/hadoop-connectors/releases/tag/v3.1.13) |
| [**Conscrypt** ^installed^](https://conscrypt.org/) | [2.6](https://github.com/google/conscrypt/tags) | [2.6](https://github.com/google/conscrypt/tags) | [2.6](https://github.com/google/conscrypt/tags) | [2.6](https://github.com/google/conscrypt/tags) | [2.6](https://github.com/google/conscrypt/tags) |
| [**Delta Lake** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/delta.md.txt) | [4.2.0](https://delta.io/blog/delta-lake-3-2/) | [4.2.0](https://delta.io/blog/delta-lake-3-2/) | [4.2.0](https://delta.io/blog/delta-lake-3-2/) |   |   |
| [**Docker** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/docker.md.txt) | [28.1](https://docs.docker.com/engine/release-notes/#version-2010) | [28.1](https://docs.docker.com/engine/release-notes/#version-2010) | [28.1](https://docs.docker.com/engine/release-notes/#version-2010) | [28.1](https://docs.docker.com/engine/release-notes/#version-2010) |   |
| [**Java** ^installed^](https://www.java.com/) | [21](https://jdk.java.net/21/) | [21](https://jdk.java.net/21/) | [21](https://jdk.java.net/21/) | [21](https://jdk.java.net/21/) | [21](https://jdk.java.net/21/) |
| [**JupyterLab Notebook** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/jupyter.md.txt) | [4.5.7](https://jupyterlab.readthedocs.io/en/stable/getting_started/changelog.html) | [4.5.7](https://jupyterlab.readthedocs.io/en/stable/getting_started/changelog.html) | [4.5.7](https://jupyterlab.readthedocs.io/en/stable/getting_started/changelog.html) | [4.5.7](https://jupyterlab.readthedocs.io/en/stable/getting_started/changelog.html) |   |
| [**Python** ^installed^](https://www.python.org/) | [Pixi 0.76.0](https://pixi.prefix.dev) with [Python 3.12](https://docs.python.org/3/whatsnew/3.12.html) | [Pixi 0.76.0](https://pixi.prefix.dev) with [Python 3.12](https://docs.python.org/3/whatsnew/3.12.html) | [Pixi 0.67.1](https://pixi.prefix.dev) with [Python 3.12](https://docs.python.org/3/whatsnew/3.12.html) | [Pixi 0.67.1](https://pixi.prefix.dev) with [Python 3.12](https://docs.python.org/3/whatsnew/3.12.html) | [Pixi 0.67.1](https://pixi.prefix.dev) with [Python 3.11](https://docs.python.org/3/whatsnew/3.11.html) |
| [**R** ^installed^](https://www.r-project.org/about.html) | [R 4.5](https://cran.r-project.org/src/base/R-4/) | [R 4.5](https://cran.r-project.org/src/base/R-4/) | [R 4.5](https://cran.r-project.org/src/base/R-4/) | [R 4.5](https://cran.r-project.org/src/base/R-4/) | [R 4.5](https://cran.r-project.org/src/base/R-4/) |
| [**Scala** ^installed^](https://www.scala-lang.org/) | [2.13.17](https://www.scala-lang.org/news/2.13.17) | [2.13.17](https://www.scala-lang.org/news/2.13.17) | [2.13.17](https://www.scala-lang.org/news/2.13.17) | [2.13.17](https://www.scala-lang.org/news/2.13.17) | [2.13.17](https://www.scala-lang.org/news/2.13.17) |
| [**Trino** ^optional component^](https://docs.cloud.google.com/managed-spark/docs/concepts/components/trino.md.txt) | [480](https://trino.io/docs/current/release/release-480.html) | [480](https://trino.io/docs/current/release/release-480.html) | [480](https://trino.io/docs/current/release/release-480.html) | [480](https://trino.io/docs/current/release/release-480.html) |   |

## Limitations

- **Metastores with Hive 4.2:** Image version `3.0` doesn't support BigQuery Metastore, BigLake Metastore, or Dataproc Metastore with Hive 4.2 (direct Hive queries using these metastores are not supported).

- **Delta Lake with Hive:** Delta Lake `4.x`, which is included in Managed Service for Apache Spark image version `3.0`, doesn't support Hive (queries on Delta Lake 4.2 tables cannot be executed on Hive).

- **Iceberg with Hive:** Managed Service for Apache Spark image version `3.0` includes Hive 4.2, which supports Iceberg through the Iceberg REST catalog.

- **GPU clusters:** All `3.0` image versions except `3.0-ml-ubuntu` enable [`cgroups V2`](https://kubernetes.io/docs/concepts/architecture/cgroups/). Since YARN doesn't support GPU discovery when `cgroups V2` is enabled, you must use the `3.0-ml-ubuntu` image when creating clusters with GPUs.

## Notes:

- Version `3.0` is a lightweight image that contains only core components,
  reducing exposure to Common Vulnerabilities and Exposures (CVEs). For higher
  security compliance requirements, use the image version `2.3`or later, when
  creating a Managed Service for Apache Spark cluster.

- If you choose to install
  [optional components](https://docs.cloud.google.com/managed-spark/docs/concepts/components/overview.md.txt) when
  creating a Managed Service for Apache Spark cluster with `3.0` image, they will be
  downloaded and installed during cluster creation. This might increase the
  cluster startup time. To avoid this delay, you can create a
  [custom image](https://docs.cloud.google.com/managed-spark/docs/guides/images.md.txt#generate_a_custom_image)
  with the optional components pre-installed. This is achieved by running
  [`generate_custom_image.py`](https://github.com/GoogleCloudDataproc/custom-images?tab=readme-ov-file#generate-custom-image)
  with the
  [`--optional-components`](https://docs.cloud.google.com/managed-spark/docs/guides/images.md.txt#run_the_code)
  flag.

  > [!NOTE]
  > **Note:** You must specify the optional components that you want to install when you create the cluster. For more information, see [Add optional components](https://docs.cloud.google.com/managed-spark/docs/concepts/components/overview.md.txt#add_optional_components).   
  > The following example shows the Google Cloud CLI command for creating a cluster with optional components:
  >
  > ```
  > gcloud dataproc clusters create CLUSTER_NAME
  >     --optional-components=COMPONENT_NAME \
  >     ... other flags
  > ```

- The following optional components are supported in 3.0 images:

  - Apache Flink
  - Apache Hive WebHCat
  - Apache Iceberg
  - Apache Pig
  - Apache Ranger
  - Apache Solr
  - Apache Zeppelin notebook
  - Apache Zookeeper
  - Delta Lake
  - Docker
  - JupyterLab Notebook
  - Trino
- `yarn.nodemanager.recovery.enabled` and HDFS Audit Logging
  are enabled by default in 3.0 images.

- [Pixi](https://docs.cloud.google.com/managed-spark/docs/concepts/versioning/%22https:/pixi.prefix.dev%22) is installed as part of the Python
  installation and is used to install Python packages instead of Conda.

- The default resource calculator for YARN has been changed from
  [DefaultResourceCalculator](https://hadoop.apache.org/docs/r3.4.1/hadoop-yarn/hadoop-yarn-common/apidocs/org/apache/hadoop/yarn/util/resource/DefaultResourceCalculator.html)
  to [DominantResourceCalculator](https://hadoop.apache.org/docs/r3.4.1/hadoop-yarn/hadoop-yarn-common/apidocs/org/apache/hadoop/yarn/util/resource/DominantResourceCalculator.html),
  which uses the dominant-resource concept to determine resource allocation,
  such as Memory and CPU allocation. This change impacts
  [Autoscaler](https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/autoscaling.md.txt#how_autoscaling_works),
  which scales based on the dominant resource usage of the cluster.

- Starting with image version `3.0`, when you create a cluster without specifying
  a machine type for a cluster node, Managed Service for Apache Spark specifies
  the node with a ranked list of flexible VM machine types, such as a ranked list of N4, N2, and E2
  series machine types, with the list optimized for resource availability.