This document provides an overview of the Google Cloud storage services that you
can use in Cluster Director. By learning the different storage solutions that
you can use in your cluster, you can select the best storage options for your
workload requirements and budget.

High-performance storage is a critical component for large-scale artificial
intelligence (AI) and high performance computing (HPC) workloads.
Cluster Director supports two distinct storage categories:

- **Shared network storage** : cluster-wide file systems and object stores that
  all nodes share to load training datasets, process data, save model
  checkpoints, and host user home directories (`/home`).

- **Node boot disk storage**: block storage at the compute instance level that
  powers the Linux operating system (OS), GPU or TPU drivers and libraries,
  and Slurm daemons on individual compute and login nodes.

## Shared network storage

Shared storage services connect across the cluster network to provide a unified
file system or object store for all cluster nodes.

### Shared file system requirements by orchestrator

How you configure shared network storage depends on the orchestrator that your
cluster uses:

- **Slurm clusters** : when you deploy a cluster with Slurm as the
  orchestrator, you must provide a shared file system for the `/home`
  directory on all compute and login nodes. To meet this requirement, you can
  configure Cluster Director to create a new Filestore or
  Google Cloud Managed Lustre instance, or you can use an existing instance.

- **Managed compute clusters** : shared network storage is optional. You can
  mount Filestore instances, Managed Lustre
  instances, or Cloud Storage buckets on each managed instance group (MIG).
  When you add or remove network storage mounts on a MIG,
  Cluster Director updates `/etc/fstab.d/ce.fstab` and mounts or
  unmounts the storage within 30 seconds. This update doesn't restart your
  compute instances.

> [!NOTE]
> **Note:** You can use both new and existing storage resources in the same cluster. If you use an existing Filestore or Managed Lustre instance, then the instance must be on the same VPC network as your cluster. Each storage resource that you define must specify either a new resource configuration or an existing resource configuration, not both.

### Supported shared storage services

Cluster Director supports the following shared network storage services:

| Storage service | Features | Recommended for |
|---|---|---|
| [Filestore](https://docs.cloud.google.com/filestore/docs/overview.md.txt) | **Overview**: Filestore is a managed, high-performance NFS file storage service. This service provides a file system interface and serves as the default option for home directories for clusters in Cluster Director. | - Home directories - General-purpose file storage - Workloads that require an NFS interface |
| [Managed Lustre](https://docs.cloud.google.com/managed-lustre/docs/overview.md.txt) | **Overview** : Managed Lustre is a high-performance, managed parallel file system that is optimized for AI and HPC applications. With low latency and full POSIX support, this service lets you migrate on-premises AI workloads to Google Cloud. <br /> **Dynamic tier support** : when you create or modify a cluster, you can create new Managed Lustre instances or use existing instances that use the [Dynamic tier](https://docs.cloud.google.com/managed-lustre/docs/performance-tiers.md.txt#dynamic_tier). If you use this tier, then your cluster must use an existing Virtual Private Cloud (VPC) network that has the `dynamic_tier_capacity` quota allocated. | - Home directories - Migration of AI or machine learning (ML) workloads to Google Cloud - Model simulations - Workloads with frequent small reads and writes - High-throughput burst workloads |
| [Cloud Storage](https://docs.cloud.google.com/storage/docs/introduction.md.txt) | **Overview**: Cloud Storage is a scalable, durable, and cost-effective object store. When you create a cluster, you can select from the following storage classes: - **[Rapid storage class](https://docs.cloud.google.com/storage/docs/storage-classes.md.txt#rapid)**: provides low latency and high throughput for AI and ML workloads. - **[Standard storage class](https://docs.cloud.google.com/storage/docs/storage-classes.md.txt#standard)**: balances cost and performance for general-purpose use cases. - **[Autoclass](https://docs.cloud.google.com/storage/docs/autoclass.md.txt)**: automatically adjusts object storage classes based on how your workloads access data to reduce costs. Through integration with Cloud Storage FUSE, you can mount Cloud Storage buckets as local file systems for model checkpoints and training data. | - High-performance data access with the Rapid storage class - Cost-effective data storage with Standard and other storage classes - Data processing and preparation - Model training data and checkpoints |

## Node boot disk storage

Each controller, login, and compute node in a cluster requires a boot disk for
the operating system (OS), GPU or TPU drivers and libraries, and system daemons.
Boot disks attach to individual nodes and don't share files across the cluster.

Cluster Director supports the following boot disk options:

| Boot disk option | Features | Recommended for |
|---|---|---|
| [Standalone boot disks](https://docs.cloud.google.com/compute/docs/disks.md.txt) | **Overview**: Cluster Director provisions a dedicated Persistent Disk or Google Cloud Hyperdisk volume for each node. This configuration is the default choice when you create a cluster. | - Standard cluster deployments - Static clusters with fixed node counts - Workloads with predictable, constant node usage |
| [Hyperdisk storage pools](https://docs.cloud.google.com/compute/docs/disks/pools.md.txt)^1^ | **Overview**: Hyperdisk storage pools let you share capacity and performance across boot disks in a specific zone to optimize costs. In Cluster Director, you can use existing Hyperdisk pools or Hyperdisk Exapools for boot disks in Slurm clusters only. Because boot disks require Hyperdisk Balanced volumes, you can't use storage pools that use Hyperdisk Throughput volumes. | - Boot disks for compute and login nodes in Slurm clusters - Clusters that scale compute nodes on demand - Workloads with shared capacity and IOPS requirements |

^1^ Only machine series that support Hyperdisk Balanced volumes can use storage pools in Cluster Director. If you use storage pools with an unsupported machine series, then the operation to create or modify the cluster fails. For more information, see [Machine series support for Hyperdisk](https://docs.cloud.google.com/compute/docs/disks/hyperdisks.md.txt#machine-type-support).

## Non-boot disks for managed compute clusters

When you create or modify a managed compute cluster, you can configure non-boot
block disks for each compute instance in a MIG. Cluster Director creates a
separate disk for each compute instance in read-write mode from a blank volume,
source image, or snapshot in the same project. On MIGs with GPU or CPU instances
that use any consumption option except Flex-start, Cluster Director
configures each non-boot disk as a
[stateful disk](https://docs.cloud.google.com/compute/docs/instance-groups/configuring-stateful-disks-in-migs.md.txt)
so that replacement compute instances reattach the same disk across rolling
updates or preemptions. On MIGs with TPU VMs or Flex-start VMs, new
non-boot disks persist for the lifetime of the MIG.

## What's next

- [Choose an orchestrator](https://docs.cloud.google.com/cluster-director/docs/orchestration.md.txt)

- [Try out Cluster Director](https://docs.cloud.google.com/cluster-director/docs/create/quickstart.md.txt)