> [!CAUTION]
> You are currently viewing version 1.13 of the Apigee hybrid documentation. **This version is end of life.** You should upgrade to a newer version. For more information, see [Supported versions](https://docs.cloud.google.com/apigee/docs/hybrid/supported-platforms.md.txt#supported-versions).

<button class="button button-with-icon cloud-select-dropdown__button button-primary" aria-haspopup="true">Version 1.13 </button>

- Supported versions:
- [v1.17 (latest)](https://docs.cloud.google.com/apigee/docs/hybrid/v1.17/cassandra-csi-backup-restore.md.txt)
- [v1.16](https://docs.cloud.google.com/apigee/docs/hybrid/v1.16/cassandra-csi-backup-restore.md.txt)
- [v1.15](https://docs.cloud.google.com/apigee/docs/hybrid/v1.15/cassandra-csi-backup-restore.md.txt)
- [v1.14](https://docs.cloud.google.com/apigee/docs/hybrid/v1.14/cassandra-csi-backup-restore.md.txt)
- [List of supported versions](https://docs.cloud.google.com/apigee/docs/hybrid/supported-platforms.md.txt#supported-versions)
- Unsupported versions:
- [v1.13](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/cassandra-csi-backup-restore.md.txt)
- [v1.12](https://docs.cloud.google.com/apigee/docs/hybrid/v1.12/cassandra-csi-backup-restore.md.txt)
- [v1.11](https://docs.cloud.google.com/apigee/docs/hybrid/v1.11/cassandra-csi-backup-restore.md.txt)
- [v1.10](https://docs.cloud.google.com/apigee/docs/hybrid/v1.10/cassandra-csi-backup-restore.md.txt)
- [v1.9](https://docs.cloud.google.com/apigee/docs/hybrid/v1.9/cassandra-csi-backup-restore.md.txt)
- [v1.8](https://docs.cloud.google.com/apigee/docs/hybrid/v1.8/cassandra-csi-backup-restore.html)
- [v1.7](https://docs.cloud.google.com/apigee/docs/hybrid/v1.7/cassandra-csi-backup-restore.html)
- [v1.6](https://docs.cloud.google.com/apigee/docs/hybrid/v1.6/cassandra-csi-backup-restore.html)
- [v1.5](https://docs.cloud.google.com/apigee/docs/hybrid/v1.5/cassandra-csi-backup-restore.html)
- [v1.4](https://docs.cloud.google.com/apigee/docs/hybrid/v1.4/cassandra-csi-backup-restore.html)
- [v1.3](https://docs.cloud.google.com/apigee/docs/hybrid/v1.3/cassandra-csi-backup-restore.html)
- [v1.2](https://docs.cloud.google.com/apigee/docs/hybrid/v1.2/cassandra-csi-backup-restore.html)
- [v1.1](https://docs.cloud.google.com/apigee/docs/hybrid/v1.1/cassandra-csi-backup-restore.html)


You can back up and restore your
[hybrid data](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/cassandra-backup-overview.md.txt#what-is-backed-up) using
[CSI](https://kubernetes.io/docs/concepts/storage/volume-snapshots/)
(Container Storage Interface) snapshots. CSI backup triggers disk snapshots taken by the underlying
storage system using the provided CSI driver. CSI backup does not need a Google Cloud Storage bucket
or a remote server to store backup data.


CSI backup is recommended for hybrid instances hosted in Google Cloud, AWS, or Azure.


This page describes the steps to use hybrid CSI backup and restore. For an overview of hybrid back
up and restore in general, see the
[Cassandra backup and restore overview](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/cassandra-backup-overview.md.txt).

> [!WARNING]
> Warning about deleting:
>
> Be careful which version or release of the datastore components you delete. You can see
> your releases with the command `helm -n APIGEE_NAMESPACE ls`. To delete a
> specific release use `helm -n APIGEE_NAMESPACE delete RELEASE-NAME`.

## Backup and restore limitations


Be aware of these limitations when using CSI backup and restore:

- The CSI driver used by the configured storage class must support CSI snapshots. See this [Kubernetes CSI driver list](https://kubernetes-csi.github.io/docs/drivers.html) for driver information.
- Not all platforms are supported. Only Google Cloud, AWS, and Azure platforms are supported.
- OpenShift Container Platform is not supported due to volume snapshot [limitations](https://docs.openshift.com/container-platform/4.12/storage/container_storage_interface/persistent-storage-csi-snapshots.html#persistent-storage-csi-snapshots-create_persistent-storage-csi-snapshots).
- Only cloud platforms are supported. On-prem platforms are not supported.
- CSI backup data and non-CSI hybrid backup data are incompatible. Non-CSI backups cannot be used with CSI restore and CSI backups cannot be used with non-CSI restore.
- CSI driver installation and functionality is the responsibility of the CSI driver vendor.
- Users are responsible for ensuring adequate cluster resources are available for provisioning CSI snapshots.
- Users are responsible for removing old snapshot data.

## Set up CSI backups

To schedule hybrid backups using CSI, perform the following steps:

1. If you have not previously set up hybrid backup:
   1. Run the following `create-service-account` command to create a Google Cloud service account (SA) with the standard [`roles/storage.objectAdmin`](https://docs.cloud.google.com/storage/docs/access-control/iam-roles.md.txt) role. This SA role allows you to write backup data to Cloud Storage. Execute the following command in the directory appropriate for your management tool:
      - **Helm charts:** `$APIGEE_HELM_CHARTS_HOME/apigee-operator/etc/`
      - **`apigeectl`:** `HYBRID_BASE_DIRECTORY/hybrid-files/`

      ```
      ./tools/create-service-account --env non-prod --dir ./service-accounts
      ```

      This command creates a single service account named
      `apigee-non-prod` for use in non-production environments and
      places the downloaded key file in the `./service-accounts`
      directory.

      > [!NOTE]
      > **Note:** If you prefer to create all the individual service accounts for a production environment, use the following command:
      >
      > ```
      > ./tools/create-service-account --env prod --dir ./service-accounts
      > ```

      For more information about Google Cloud service accounts, see
      [Creating
      and managing service accounts](https://docs.cloud.google.com/iam/docs/creating-managing-service-accounts).
   2. The `create-service-account` command saves a JSON file containing the service account private key. The file is saved in the same directory where the command executes. You will need the path to this file in the following steps.
2. Open your `overrides.yaml` file. Set the follow parameters, as shown in [Example overrides files](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/cassandra-csi-backup-restore.md.txt#example-overrides-files).

   <br />

   1. Set the general parameters shown below in the `backup` block. If you've already set these parameters for the non-CSI hybrid back up solution, you can use the same parameters for your CSI snapshots. See [backup properties reference table](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/schedule-cassandra-backup-gcs.md.txt#backup-properties-ref) for more information about each value.  


      For `backup`:
      - **enabled:** Set to `true` to enable scheduled backups.
      - **pullPolicy in image:** Set to `Always`.
      - **schedule:** Provide a [cron expression schedule](https://en.wikipedia.org/wiki/Cron#Cron_expression).
   2. Set these parameters for CSI-specific backup:
      - **Cassandra storage group values:** The configured Cassandra storage class must support CSI snapshots for CSI backup and restore to work. To check if a storage class supports CSI snapshots, run the following command to get the available storage classes:

        ```
        kubectl get sc
        ```
        Look at the "Provisioner" output for each storage class. Provisioners using CSI usually have a ".csi." part to their name like "pd.csi.storage.gke.io". Look for the provisioner name in this [Kubernetes CSI driver list](https://kubernetes-csi.github.io/docs/drivers.html). If the "Other Features" column for the provisioner contains the word "SNAPSHOT", then the storage class using the provisioner supports CSI snapshots.

        Add these parameters in the storage group. Both values are required.
        - **storageclass:** A CSI snapshot-enabled storage class name.
        - **capacity:** The capacity of the disk.
      - **Cloud provider type:**

        Once the CSI snapshot capability has been verified, modify the overrides file to use
        CSI backup and restore:
        - **cloudProvider:** Set `cloudProvider` in `backup` and `restore` to `CSI`.

### Example backup config

This section shows the backup-related portions of an example `overrides.yaml` file.

> [!NOTE]
> **Note:** If your initial Apigee Hybrid installation was configured without the ` cassandra.storage ` section in your overrides file, you should not add the ` cassandra.storage ` section in your overrides file. Adding this section post-installation to enable backups can lead to an upgrade failure as this may be interpreted as an attempt to modify the storage class of the existing Cassandra statefulset which is not a supported operation.

```
cassandra:
  hostNetwork: false
  replicaCount: 3
  storage:
    storageclass: standard-rwo
    capacity: 100Gi
  image:
    pullPolicy: Always

  backup:
    enabled: true
    image:
      pullPolicy: Always
    cloudProvider: "CSI"
    schedule: "0 * * 11 *"
```

## Launch a manual backup


CSI backups generate automatically according to the cron schedule set in the `overrides.yaml` file.


To initiate a manual CSI backup, use this command:

```
kubectl create job -n APIGEE_NAMESPACE --from=cronjob/apigee-cassandra-backup BACKUP_POD_NAME
```
where `BACKUP_POD_NAME` is the name of the backup pod that will be created.

<br />

## Verify backups


One way to verify a backup was successfully created is to check the volume snapshots on the
Kubernetes cluster, using this command:

```
kubectl get volumesnapshot -n APIGEE_NAMESPACE
```


The output shows the current list of snapshots on the cluster. The CSI backup process creates a
snapshot of each Cassandra disk. The number of generated snapshots should match the total number of
cassandra pods in the cluster.

## Restore a backup


Use this process to restore a previously generated CSI backup. For general information on
restoring backups and an overview of the process, see the
[restore overview page](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/restore-cassandra-backup.md.txt).


To initiate a restore of a CSI backup, follow the instructions for the
[hybrid non-CSI single region restore](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/restore-cassandra-single-region.md.txt),
but use these values in the `restore` block in your `overrides.yaml`.
See the [backup properties reference table](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/schedule-cassandra-backup-gcs.md.txt#backup-properties-ref)
for more information about each value and the
[example restore configuration](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/cassandra-csi-backup-restore.md.txt#example-restore-config) for an example.

- **enabled:** Set to `true` to enable restore for the backup referenced with the `snapshotTimestamp` timestamp.
- **snapshotTimestamp:** Provide the timestamp of a previous CSI backup.
- **pullPolicy in image:** Set to `Always`.

To find the `snapshotTimestamp` value to restore, run this command to get the
list of available snapshots:

```
kubectl get volumesnapshot -n APIGEE_NAMESPACE
```
In the returned list, the names of the snapshots contain the timestamp:

```
pvc-us-west2-b-20220803004907-47beff0e306d8861
```
In this example the timestamp is `20220803004907`.

<br />

Important: Terminating the Cassandra cluster is a normal part of the CSI restoration process. After terminating the cluster, the restore process creates new PVCs containing the backup data and then re-creates the Cassandra cluster using the new PVCs. Do not be concerned if you see the cluster terminating.

### Example restore config

This section shows the restore-related portions of an example `overrides.yaml` file.

```
cassandra:
  hostNetwork: false
  replicaCount: 3
  storage:
    storageclass: standard-rwo
    capacity: 100Gi
  image:
    pullPolicy: Always

  restore:
    enabled: true
    snapshotTimestamp: "20220908222130"
    cloudProvider: "CSI"
    image:
      pullPolicy: Always
```

## Migrate to CSI backup and restore


If you have not previously used hybrid backup and restore, you can follow the instructions in
[Set up CSI backups](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/cassandra-csi-backup-restore.md.txt#set-up-csi-backups) to create a new CSI backups
without the steps in this section. These steps guide you through migrating from the non-CSI
backup and restore solution to CSI backups.

1. Generate a new backup using the currently configured non-CSI backup method.
2. Change the backup configuration in the hybrid `overrides.yaml` file to use the CSI backup overrides as shown in the [](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/cassandra-csi-backup-restore.md.txt#example-backup-config)example backup config.
3. Apply the changes in the `overrides.yaml` file:

   ```
   helm upgrade datastore apigee-datastore/ \
     --namespace APIGEE_NAMESPACE \
     --atomic \
     -f OVERRIDES_FILE.yaml
   ```

   > [!NOTE]
   > **Note:** If you see an error saying `Error: UPGRADE FAILED: "datastore" has no deployed releases`, replace `upgrade` with `install` and try the command again.

4. Verify the backup job:

   ```
   kubectl get cronjob -n APIGEE_NAMESPACE
   ```
5. After a backup job completes, verify snapshots have been created. The number of generated snapshots should be equivalent to the number of Cassandra nodes in the Hybrid instance.

   ```
   kubectl get volumesnapshot -n APIGEE_NAMESPACE
   ```