Managed Service for Apache Spark lets you store and process workload data by using the Hadoop Distributed File System (HDFS) and Cloud Storage in parallel. With the pre-installed [Cloud Storage connector](https://docs.cloud.google.com/managed-spark/docs/concepts/connectors/cloud-storage.md.txt), you can transfer data into and out of your clusters while tailoring underlying VM disk storage to your job performance requirements.

## Storage options and considerations

The following features and considerations can be important
when selecting compute and data storage options for Managed Service for Apache Spark
clusters and jobs:

- HDFS with Cloud Storage: Managed Service for Apache Spark uses the Hadoop Distributed File System (HDFS) for storage. Additionally, Managed Service for Apache Spark automatically installs the HDFS-compatible [Cloud Storage connector](https://docs.cloud.google.com/managed-spark/docs/concepts/connectors/cloud-storage.md.txt), which enables the use of Cloud Storage in parallel with HDFS. Data can be moved in and out of a cluster through upload and download to HDFS or Cloud Storage.
- VM disks:
  - By default, when no local SSDs are provided, HDFS data and intermediate shuffle data is stored on VM boot disks, which are [Persistent Disks](https://docs.cloud.google.com/persistent-disk).
  - If you use [local SSDs](https://docs.cloud.google.com/managed-spark/docs/concepts/compute/dataproc-local-ssds), HDFS data and intermediate shuffle data is stored on the SSDs.
  - Persistent disk (PD) size and type affect performance and VM size, whether using HDFS or Cloud Storage for data storage.
  - **VM Boot disks are deleted when the cluster is deleted.**