Customer-managed encryption keys for Managed Lustre

By default, Google Cloud Managed Lustre encrypts customer content at rest. Managed Lustre handles encryption for you without any additional actions on your part. This option is called Google default encryption.

If you want to control your encryption keys, then you can use customer-managed encryption keys (CMEKs) in Cloud KMS with CMEK-integrated services including Managed Lustre. Using Cloud KMS keys gives you control over their protection level, location, rotation schedule, usage and access permissions, and cryptographic boundaries. Using Cloud KMS also lets you track key usage, view audit logs, and control key lifecycles. Instead of Google owning and managing the symmetric key encryption keys (KEKs) that protect your data, you control and manage these keys in Cloud KMS.

After you set up your resources with CMEKs, the experience of accessing your Managed Lustre resources is similar to using Google default encryption. For more information about your encryption options, see Customer-managed encryption keys (CMEK).

The following Managed Lustre data is protected by CMEK:

File data
File system metadata, such as filenames

When you use CMEK in Managed Lustre, your projects can consume Cloud KMS cryptographic requests quotas. CMEK-encrypted instances consume quotas when reading or writing data in Managed Lustre. Encryption and decryption operations using CMEK keys affect Cloud KMS quotas only if you use hardware (Cloud HSM) or external (Cloud EKM) keys. For more information, see Cloud KMS quotas.

Limitations

CMEK-protected Managed Lustre instances cannot be used with Google Kubernetes Engine or Cluster Toolkit. They can only be accessed from Compute Engine clients.

Create a Cloud KMS key

Before creating a CMEK-enabled Lustre instance, you must have a Cloud KMS key ring and a Cloud KMS key.

Managed Lustre supports three protection levels:

Software (SOFTWARE)
Hardware (HSM)
External over the internet (EXTERNAL)

When creating your key, we recommend creating your Cloud KMS key in a different project from the project containing your Managed Lustre instance. See Separation of duties.

The Cloud KMS key must be created in the same region as the Managed Lustre instance you are creating.

To create a key:

If you don't already have one, create a key ring.
Create a Cloud KMS key.

Grant IAM permissions to the Managed Lustre service agent

Managed Lustre uses a service agent to access your resources and perform actions on your behalf. You grant the service agent IAM roles to manage its access. The Managed Lustre service agent is automatically created when you create a Managed Lustre instance.

The service agent identity is of the form service-PROJECT_NUMBER@gcp-sa-lustre.iam.gserviceaccount.com.

Create or retrieve the service agent

If you don't already have a Managed Lustre service agent

If you've never created a Managed Lustre instance in the project, run the following command to create the service agent manually:

gcloud beta services identity create \
  --service=lustre.googleapis.com \
  --project=PROJECT_NUMBER_OR_ID

Replace PROJECT_NUMBER_OR_ID with the project number or ID of the project where you want to create the Managed Lustre instance. The output is similar to the following:

Service identity created: service-1234567890@gcp-sa-lustre.iam.gserviceaccount.com

Copy the value of the service agent identity to use in the next step.

If you've already created a Managed Lustre instance

To construct the service agent identity, obtain your project number. A PROJECT_NUMBER is not the same as a project ID:
- A project ID is a unique string that can be a combination of letters, numbers, and hyphens. You specify a project ID when creating your project. For example, example-project-123.
- A project number is an automatically generated unique identifier for your project that consists only of numbers. For example, 1234567890.
To obtain the PROJECT_NUMBER for a given project ID, use the gcloud projects describe command:
```
gcloud projects describe PROJECT_ID --format="value(projectNumber)"
```
Copy the project number that is returned into the service agent identity:
```
service-PROJECT_NUMBER@gcp-sa-lustre.iam.gserviceaccount.com
```
Copy the service agent identity to use in the next step.

Grant permissions to the service agent

Managed Lustre needs authorization to perform operations with your key.

You must grant the Managed Lustre service agent the roles/cloudkms.cryptoKeyEncrypterDecrypter IAM role on your Cloud KMS key.

Use the gcloud kms keys add-iam-policy-binding command:

gcloud kms keys add-iam-policy-binding KEY_NAME \
  --keyring=KEY_RING \
  --location=REGION \
  --project=KMS_PROJECT_ID \
  --member=serviceAccount:service-SERVICE_AGENT_IDENTITY \
  --role=roles/cloudkms.cryptoKeyEncrypterDecrypter

Replace the following variables:

KEY_NAME: The name of your Cloud KMS key.
KEY_RING: The name of the key ring that contains the key.
REGION: The region where your key ring is located.
KMS_PROJECT_ID: The ID of the project that contains your Cloud KMS key.
SERVICE_AGENT_IDENTITY: The Managed Lustre service agent identity from the previous step, in the format: service-PROJECT_NUMBER@gcp-sa-lustre.iam.gserviceaccount.com.

Create a CMEK-enabled Managed Lustre instance

To create a Managed Lustre instance protected by CMEK, specify the resource identifier of the key during instance creation using the Google Cloud CLI command:

gcloud lustre instance create INSTANCE_ID \
  --capacity-gib=CAPACITY_GIB \
  --location=ZONE \
  --network=NETWORK_NAME \
  --project=PROJECT_ID \
  --kms-key-name=projects/KMS_PROJECT_ID/locations/REGION/keyRings/KEY_RING/cryptoKeys/KEY_NAME

For a description of all available fields, see Create a Managed Lustre instance.

If the Cloud KMS key isn't available, or there are permission issues when you initiate the create operation, the operation fails and returns a FAILED_PRECONDITION error.

Key management

Cloud Key Management Service keys can be rotated, disabled, re-enabled, or destroyed. They can become unavailable for various reasons.

Rotate a key

You can rotate your Cloud KMS key. Rotating a key creates a new key version.

Rotating a key does not re-encrypt the data in your Managed Lustre instance. Managed Lustre does not support decrypting or re-encrypting data.
The existing capacity of your Managed Lustre instance continues to be encrypted by the version of the key in use when the instance was created.
When you increase your instance's storage capacity, the new capacity is protected by the key version that is primary when you make the capacity increase request, and any data that is written to that capacity is protected by that key version.

This means that you can have multiple key versions encrypting your Managed Lustre instance, if you have increased the instance's capacity after rotating the key version.

Disable a key

You can disable a key version. During the time that any key version used by your Managed Lustre instance is disabled, the instance is suspended. To resume your instance, return all key versions that protect your instance to an available state. See Unavailable keys for information about suspended instances.

See Enable and disable key versions for more information.

Re-enable a key

If a key version is disabled, you can re-enable it through the Cloud KMS API. See Enable and disable key versions.

To resume the Managed Lustre instance, all versions of the key that are encrypting storage capacity must be enabled or restored.

After all required keys return to the ENABLED state, Managed Lustre automatically detects key changes and restarts the instance. The instance state becomes ACTIVE and the Managed Lustre service resumes normal operation. You can perform I/O and all other long-running operations.

Destroy a key

Cloud KMS key versions can be destroyed, meaning that the key material is permanently deleted.

Cloud KMS keys cannot be destroyed immediately; instead, you must schedule a key version for destruction. During the scheduled duration, you can restore a key version to cancel its destruction. See Destroy and restore key versions for more details.

A key version that is scheduled for destruction is unavailable; any Managed Lustre instances protected by that key version are suspended. See Unavailable keys for information about suspended instances.

Once a key version is destroyed, it cannot be restored. Your instance remains suspended permanently. The only operation available on the Managed Lustre instance is deleting the instance.

Unavailable keys

A key can become unavailable for the following reasons:

The key is disabled.
The key is destroyed or scheduled to be destroyed.
The required permissions are removed from the service agent.
Billing is disabled on the Cloud KMS key's project and remains disabled for more than one hour.
An external key remains unreachable for more than one hour.

In these cases, within a few minutes of the event, the Managed Lustre instance is shut down and its instance state is set to SUSPENDED.

A SUSPENDED instance can be recovered by making the key available again.

Possible suspension reasons are as follows:

Action on key	Instance state	Suspension reason	Can be re-enabled
Disable	`SUSPENDED`	`KEY_DISABLE`	Yes
Schedule for destruction	`SUSPENDED`	`KEY_DESTROY_SCHEDULED`	Yes
Destroy	`SUSPENDED`	`KEY_DESTROY`	No
Remove permission	`SUSPENDED`	`PERMISSION_DENIED`	Yes
Disable billing for 1h	`SUSPENDED`	`BILLING_DISABLED`	Yes
EKM key unreachable for 1h	`SUSPENDED`	`EKM_KEY_UNREACHABLE`	Yes

When an instance is suspended:

Long-running operations, such as instance updates and import/export, are blocked and return a FAILED_PRECONDITION error.
I/O operations stop responding until server nodes are shut down. This occurs within 30 minutes.
The only allowed long-running operation is DeleteInstance.

Instances that have been suspended for longer than four months may not be able to be resumed.

Billing for suspended instances

If a Managed Lustre instance enters the SUSPENDED state due to key unavailability, you continue to accrue charges for the instance. The suspension state does not pause billing. To stop incurring charges for the instance, you must delete it.

You are also billed separately by Cloud KMS for any key versions in use.

View key usage and list key versions

You can view the Google Cloud resources within your organization that are protected by your Cloud KMS keys. See View key usage for details.

To view the full list of key versions used by your Managed Lustre instance, use the gcloud kms inventory search-protected-resources command:

gcloud kms inventory search-protected-resources \
  --keyname=projects/KMS_PROJECT_ID/locations/REGION/keyRings/KEY_RING/cryptoKeys/KEY_NAME \
  --scope=organizations/ORGANIZATION_ID \
  --filter="name:projects/PROJECT_ID/locations/REGION/instances/INSTANCE_ID" \
  --flatten="cryptoKeyVersions" \
  --format="value(cryptoKeyVersions)"

Where:

ORGANIZATION_ID is the numerical ID of your organization. See Getting your organization resource ID.

You can also view key information from the Key Inventory page.

Go to Key Inventory

Customer-managed encryption keys for Managed Lustre Stay organized with collections Save and categorize content based on your preferences.