You create persistent storage for workloads running on
GKE on AWS with Kubernetes objects.

## Using persistent storage in your GKE on AWS workloads

In GKE on AWS, you use the Kubernetes
[PersistentVolume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/)
(PV),
[PersistentVolumeClaim](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#lifecycle-of-a-volume-and-claim)
(PVC), and
[StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/)
resources to provide persistent file and block storage to workloads. For user clusters version 1.17 and higher, you can create snapshots of persistent storage with the
[VolumeSnapshot](https://kubernetes.io/docs/concepts/storage/volume-snapshots/)
and [VolumeSnapshotClass](https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes/) resources.
Snapshots are stored in your AWS account.

### StorageClass

User clusters have a
[default](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#class-1)
Kubernetes StorageClass that
[dynamically provisions](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes.md.txt#dynamic_provisioning)
stateful storage for workloads on AWS Elastic Block Storage (EBS) volumes. You
can also use a
[different StorageClass](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/storage-class.md.txt#custom) to
provision other types of volumes. These options are described in the following
section.

### VolumeSnapshotClass

Users clusters have a
[default](https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes/#the-volumesnapshotclass-resource)
Kubernetes VolumeSnapshotClass that creates snapshots of stateful storage on AWS
Elastic Block Storage (EBS) volumes.

## Configuring cluster storage

If you want to provision storage volumes with a non-default StorageClass,
you can create a new
[StorageClass](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/storage-class.md.txt#custom) in a cluster that uses
different parameters or a different storage driver. Then, you can
[set the StorageClass as the cluster's default](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/storage-class.md.txt#set_default),
or configure your workloads to use the StorageClass. For example, you can use a
[StatefulSet](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/storage-class.md.txt#statefulset) to set a specific
StorageClass name.

## Using volume snapshots

In Kubernetes versions 1.17+, you can use the
[VolumeSnapshot](https://kubernetes.io/docs/concepts/storage/volume-snapshots/)
resource to create snapshots of storage volumes. Then, you can
[provision](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#volume-snapshot-and-restore-volume-from-snapshot-support)
new persistent volumes from these snapshots.

## Elastic Block Storage (EBS)

GKE on AWS manages [AWS EBS](https://aws.amazon.com/ebs/)
volumes with the
[aws-ebs-csi-driver](https://github.com/kubernetes-sigs/aws-ebs-csi-driver).

The EBS CSI Driver version is tied to a GKE on AWS Kubernetes
version. When you upgrade your user cluster version, the new driver
version applies automatically. This version is typically the latest driver
available at the time that the GKE on AWS version is released.

The driver pre-installed with GKE on AWS provides the following
StorageClasses by default:

- `standard-rwo` (default): Used for provisioning EBS `gp2` volumes.
- `premium-rwo`: Used for provisioning EBS `io1` volumes.

EBS volumes support the Kubernetes ReadWriteOnce access mode.

> [!NOTE]
> **Note:** The legacy Kubernetes `kubernetes.io/aws-ebs` volume type is not supported.

### Preexisting EBS volumes

You can provision existing EBS volumes in your GKE on AWS
workloads. See
[Importing a preexisting EBS volume](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/preexisting-volume.md.txt) for
examples on how to use EBS volumes in GKE on AWS.

## Elastic File System (EFS)

GKE on AWS 1.6 and higher supports mounting preexisting
[AWS EFS](https://aws.amazon.com/efs/) file systems
and access points. For more information, see
[Using EFS](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/using-efs.md.txt).

## Additional storage options

Other third-party storage systems can be used with GKE on AWS
through third-party Container Storage Interface (CSI) drivers (recommended) or
in-tree Kubernetes volume plugins.

> [!NOTE]
> **Note:** Google does not provide support for third-party drivers. Contact your storage vendor for support.

### CSI drivers

The
[Container Storage Interface (CSI)](https://github.com/container-storage-interface/spec/blob/master/spec.md)
is an open standard API that enables Kubernetes to attach arbitrary storage
systems to containerized workloads. A non-comprehensive list of CSI drivers can
be found in the
[Kubernetes CSI Developer Documentation](https://kubernetes-csi.github.io/docs/drivers.html).
GKE on AWS supports CSI v1.x.

To use a CSI driver in your cluster, you need to
[install the CSI driver](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/storage-drivers.md.txt)
provided by your storage vendor. You can then configure workloads to use the
driver's StorageClass or set it as the
[default StorageClass](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/storage-class.md.txt#set_default).

### AWS-specific storage drivers

The following volume drivers can be used with GKE on AWS.

#### FSx for Lustre (FSX)

GKE on AWS doesn't manage
[FSx](https://aws.amazon.com/fsx/lustre) volumes directly.
You can manually install the
[aws-fsx-csi-driver](https://github.com/kubernetes-sigs/aws-fsx-csi-driver),
but support is not available from Google.

### Kubernetes in-tree volume plugins

Kubernetes ships with
[in-tree (built-in) volume plugins](https://kubernetes.io/docs/concepts/storage/volumes/#volume-types).
Supported in-tree drivers include the following:

- configMap
- emptyDir
- hostPath
- nfs
- projected
- secrets

> [!NOTE]
> **Note:** Using a vendor-supported in-tree volume plugin is not recommended. New Kubernetes storage features, such as volume snapshots, are supported only with CSI drivers. Also, many of the in-tree plugins do not support [dynamic provisioning](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes.md.txt). When using a driver that does not support dynamic provisioning, you must manually create PersistentVolumes for your workloads.

## What's next?

- Learn about [Using StorageClasses with your workloads](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/storage-class.md.txt).
- [Import a preexisting EBS volume into GKE on AWS](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/preexisting-volume.md.txt).
- Learn about [Persistent volumes in GKE](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes.md.txt).
- Learn about [Volume snapshots in GKE](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/volume-snapshots.md.txt).
- Read the Kubernetes [Storage Classes](https://kubernetes.io/docs/concepts/storage/storage-classes/) documentation.
- Read the [Container Storage Interface](https://kubernetes.io/blog/2019/01/15/container-storage-interface-ga/) release announcement.