Back up instance data

If data on a Google Cloud Managed Lustre instance is accidentally deleted or corrupted, or if an instance is deleted, that data is not recoverable directly on the instance. Managed Lustre does not support undeleting files or recovering instances after deletion.

To protect against data loss, you should periodically back up your instance data by exporting it to a Cloud Storage bucket.

Backup strategy with Cloud Storage

You can use Managed Lustre's built-in data transfer capability to export data from your instance directly to a Cloud Storage bucket.

For instructions, see Transfer data to Cloud Storage.

Key considerations for your backup strategy:

• Incremental exports: Transfers from Managed Lustre to Cloud Storage are incremental. They only copy files that don't already exist in the destination bucket or that have changed since the last transfer.

• Deleted files: Exports do not delete files in the destination bucket that have been deleted from the instance.

• Hierarchical namespace (HNS): For faster transfer speeds, consider using a Cloud Storage bucket with hierarchical namespace enabled.

• Storage classes: To manage storage costs for long-term backups, consider using Cloud Storage storage classes (such as Nearline, Coldline, or Archive) or configuring Object Lifecycle Management on your backup bucket. For details, see Cloud Storage pricing.

• Bucket location: To avoid cross-region data transfer charges and minimize latency during backup and restore operations, create your backup bucket in the same region as your Managed Lustre instance.

Automate periodic backups

To maintain regular backups, you can automate periodic data exports. For example, you can use:

• Cloud Scheduler: Trigger exports on a recurring schedule by sending HTTP requests to the instances.exportData REST API.
• Cron jobs: Run scheduled scripts on a client Compute Engine VM or management instance using the gcloud lustre instances export-data command.

Restore data

To restore data from a Cloud Storage backup bucket, you import the data back into a Managed Lustre instance. Data import transfers are incremental, copying only files that don't already exist on the target instance or that have changed since the last transfer. If a file exists on both the instance and the backup bucket, the import overwrites the file on the instance with the backup version if the file has been modified.

For detailed transfer instructions and commands, see Transfer data to or from Cloud Storage.

Depending on your recovery scenario, you can restore data to an existing instance or recreate an instance from a backup:

• Restore data to an existing instance:

  • Used to recover deleted or corrupted files, or to restore specific directories to an active instance.
  • You can restore an entire file system or specify a subdirectory path. If restoring to a subdirectory, that directory must already exist on the file system.
  • If a file on the instance has been modified or corrupted, importing the backup overwrites that file with the version from Cloud Storage.
  • To prevent write conflicts or data inconsistency during the import, pause client write activity or unmount clients from the target directory path.

• Recreate an instance from a backup:

  • Used for disaster recovery or after an instance is accidentally deleted.
  • Create a new Managed Lustre instance with your chosen capacity, performance tier, and VPC network.
  • Import the backup data from your Cloud Storage bucket into the root directory (/) of the new instance.
  • After the import operation completes, update your client mount configurations on Compute Engine or update your Google Kubernetes Engine PersistentVolume manifests with the new instance's IP address and file system name.