About vSAN encryption

Encryption of vSAN data at rest requires a key management system (KMS). By default, key management for vSAN data encryption in Google Cloud VMware Engine uses Cloud Key Management Service for new private clouds at no additional cost.

You can deploy an external KMS for encryption of vSAN data at rest from the following supported vendors. This page explains vSAN encryption behavior and summarizes how to use an external KMS to encrypt virtual machine data at rest in VMware Engine.

If you use a third-party KMS solution, you must provide the required licenses.

vSAN data encryption

By default, VMware Engine enables vSAN encryption for data in the primary cluster and in clusters that you subsequently add to the private cloud. Encryption of vSAN data at rest uses a data encryption key (DEK) stored on the local physical disk of the cluster after encryption. ESXi hosts automatically generate the DEK, which is a FIPS 140-2 Level 1 compliant AES-256-bit encryption key. A key encryption key (KEK) supplied by the Google-owned and managed key provider encrypts the DEK.

Google strongly recommends that you don't disable vSAN encryption of data at rest, because doing so can violate the service-specific terms for Google Cloud VMware Engine. If you disable vSAN encryption of data at rest on a cluster, the VMware Engine monitoring logic raises an alert. To help prevent you from violating the service terms, this alert triggers a Cloud Customer Care-driven action to re-enable vSAN encryption on the affected cluster.

Similarly, if you configure an external KMS, Google strongly recommends that you don't delete the key provider configuration of Cloud Key Management Service in vCenter Server.

Default key provider

VMware Engine configures vCenter Server in new private clouds to connect to a Google-owned and managed key provider. VMware Engine creates one instance of the key provider per region, and the key provider uses Cloud KMS to encrypt the KEK. VMware Engine fully manages the key provider and configures it to be highly available in all regions.

The Google-owned and managed key provider complements the built-in key provider in vCenter Server (in vSphere 7.0 Update 2 and later) and is the recommended approach for production environments. The built-in key provider runs as a process within the vCenter Server, which runs on a vSphere cluster in VMware Engine. VMware recommends against using the built-in key provider for encrypting the cluster that hosts vCenter Server. Instead, use the Google-managed default key provider or an external KMS.

Key rotation

When you use the default key provider, you are responsible for rotating the KEK. To rotate the KEK in vSphere, see the VMware documentation Generate New Data-At-Rest Encryption Keys.

For more ways to rotate a key in vSphere, see the following VMware resources:

Requirement for encrypted virtual machines

You can manage encryption keys for VMs by using the default Google-owned and managed key provider or Cloud Key Management Service.

If you enable VM Encryption (or vTPM) for any VMs in your private cloud and use a KMS to manage encryption keys, you must re-encrypt (shallow re-key) each VM after you rotate your KMS key.

A shallow re-key replaces only the key encryption key (KEK) and doesn't change the data encryption key (DEK) of the VMs. You typically trigger a shallow re-key by using the Re-encrypt action in the vSphere Client.

During this operation, the system re-wraps (re-encrypts) the existing DEK by using a new KEK. This process is fast because it doesn't rewrite actual data on the disk; instead, it only updates the small key bundle that contains the encrypted DEK. For more information, see the following VMware documentation:

Risks of failing to re-key encrypted VMs

Failing to re-key encrypted VMs before you delete the rotated (old) KMS key version can cause the following issues:

  • Failed vMotions: ESXi hosts can't decrypt VM DEKs during vMotion if you reboot the destination hosts or add them to the cluster after the KMS key rotation but before you perform the VM re-key.
  • Power-on failures: If a host reboots or clears its local key cache, it can't reacquire keys from the KMS. If you deleted the required keys from the KMS, the host can't decrypt the DEK, which prevents encrypted VMs from powering on.

Steps to perform a re-key operation on workload VMs

  1. In the vSphere Client, right-click the VM.
  2. Select VM Policies > Re-encrypt.
  3. Confirm the re-encryption request in the dialog that appears.
  4. Wait for the task to complete.
  5. Verify the re-key by migrating the VM to a host that you rebooted or added to the cluster after the KMS key rotation.

Supported vendors

To switch your active KMS, you can select a third-party KMS solution that is KMIP 1.1 compliant and certified by VMware for vSAN. The following vendors validated their KMS solutions with VMware Engine and published deployment guides and support statements:

For configuration instructions, see the following documents:

Use a supported vendor

Each deployment of an external KMS requires the same basic steps:

  • Create a Google Cloud project or use an existing one.
  • Create a new Virtual Private Cloud (VPC) network or choose an existing VPC network.
  • Connect your selected VPC network to the VMware Engine network.

Then, deploy the KMS in a Compute Engine VM instance:

  1. Set up required IAM permissions to deploy Compute Engine VM instances.
  2. Deploy the KMS in Compute Engine.
  3. Establish trust between vCenter and the KMS.
  4. Enable vSAN data encryption.

The following sections briefly describe this process of using one of the supported vendors.

Set up IAM Permissions

You need sufficient permissions to deploy Compute Engine VM instances in a given Google Cloud project and VPC network, connect your VPC network to VMware Engine, and configure firewall rules for the VPC network.

Project owners and IAM principals with the Network Admin role can create allocated IP ranges and manage private connections. For more information on roles, see Compute Engine IAM roles.

Deploy key management system in Compute Engine

Some KMS solutions are available in an appliance form factor in Google Cloud Marketplace. You can deploy these appliances by importing the OVA directly in your VPC network or Google Cloud project.

For a software-based KMS, deploy a Compute Engine VM instance by using the configuration (vCPU count, vMem, and disks) that the KMS vendor recommends. Install the KMS software in the guest operating system. Create the Compute Engine VM instance in a VPC network connected to the VMware Engine network.

Establish trust between vCenter and the KMS

After you deploy the KMS in Compute Engine, configure your VMware Engine vCenter to retrieve encryption keys from the KMS.

To retrieve encryption keys, establish trust between vCenter and your KMS by doing the following:

  1. Generate a certificate in vCenter.
  2. Sign the certificate by using a token or key that the KMS generates.
  3. Upload the signed certificate to vCenter.
  4. Verify the connectivity status on the vCenter Server configuration page.

Enable vSAN data encryption

In vCenter, the default CloudOwner user has sufficient privileges to enable and manage vSAN data encryption.

To switch from an external KMS back to the default Google-owned and managed key provider, follow the steps for changing the key provider provided in the VMware documentation Configuring and Managing a Standard Key Provider.

What's next