GKE on AWS uses the AWS Key Management Service (KMS) to encrypt
EBS volumes. GKE on AWS 1.7 and higher supports encrypting volumes
with customer managed
[Customer Master Keys](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#customer-cmk)
(CMKs). GKE on AWS versions 1.6.x and lower support
[AWS Managed Customer Master Keys](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#aws-managed-cmk) only.

This topic describes how to set up customer managed CMK to encrypt volumes. If
you don't set up customer managed CMK, GKE on AWS uses AWS managed
CMKs by default.

## Overview

The following GKE on AWS components support customer managed
CMK encrypted volumes:

- Bastion host root volume
- Management service root volumes
- Management service etcd data volumes
- User cluster control plane root volumes
- User cluster control plane etcd data volumes
- User cluster node root volumes

You specify keys in your management service, cluster, and node pool
configuration.

## Prerequisites

To use customer managed CMK for GKE on AWS root volumes, you need the following:

- Permissions to create or use an AWS KMS key policy. For more information, see [Using key policies in AWS KMS](https://docs.aws.amazon.com/kms/latest/developerguide/key-policies.html).

## Create a key policy

Your AWS KMS keys must have a key policy that allows GKE on AWS to
create and read volumes encrypted with a customer managed CMK. An example policy
is provided in the following section. To create the policy, perform the
following the steps:

1. From your `anthos-aws` directory, use
   `terraform` to get the ID of your management service.

   ```sh
   cd anthos-aws
   terraform output cluster_id
   ```
   The output includes your management service ID. In the example below, the ID is `gke-12345abc`.

       terraform output cluster_id
       gke-12345abc

   <br />

2. [Create an AWS KMS key policy](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html)
   with the following contents:

   ```json
   {
       "Version": "2012-10-17",
       "Id": "key-consolepolicy-3",
       "Statement": [
           {
               "Sid": "Enable IAM User Permissions",
               "Effect": "Allow",
               "Principal": {
                   "AWS": "arn:aws:iam::ACCOUNT_NUMBER:root"
               },
               "Action": "kms:*",
               "Resource": "*"
           },
           {
               "Sid": "Allow creating encrypted EBS volumes for EC2 use",
               "Effect": "Allow",
               "Principal": {
                   "AWS": [
                       "arn:aws:iam::ACCOUNT_NUMBER:role/gke-CLUSTER_ID-management",
                       "arn:aws:iam::ACCOUNT_NUMBER:user/AWS_USER",
                       "arn:aws:iam::ACCOUNT_NUMBER:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling"
                   ]
               },
               "Action": [
                   "kms:GenerateDataKeyWithoutPlaintext",
                   "kms:Decrypt"
               ],
               "Resource": "*",
               "Condition": {
                   "StringEquals": {
                       "kms:CallerAccount": "ACCOUNT_NUMBER",
                       "kms:ViaService": "ec2.AWS_REGION.amazonaws.com"
                   }
               }
           },
           {
               "Sid": "Allow attaching encrypted EBS volumes for EC2 Use",
               "Effect": "Allow",
               "Principal": {
                   "AWS": [
                       "arn:aws:iam::ACCOUNT_NUMBER:role/CLUSTER_ID-management",
                       "arn:aws:iam::ACCOUNT_NUMBER:role/CLUSTER_ID-controlplane",
                       "arn:aws:iam::ACCOUNT_NUMBER:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling"
                   ]
               },
               "Action": [
                   "kms:CreateGrant",
                   "kms:ListGrants",
                   "kms:RevokeGrant"
               ],
               "Resource": "*",
               "Condition": {
                   "StringEquals": {
                       "kms:CallerAccount": "ACCOUNT_NUMBER",
                       "kms:ViaService": "ec2.AWS_REGION.amazonaws.com"
                   }
               }
           }
       ]
   }
   ```

   Replace the following:
   - `ACCOUNT_NUMBER`: your AWS account number--- for example, `1234567890`.
   - `CLUSTER_ID`: your GKE on AWS management cluster ID--- for example `gke-12345678`.
   - `AWS_USER`: your AWS user name.
   - `AWS_REGION`: The AWS region where your GKE on AWS clusters run--- for example `us-east1`.

## Configuring resources to use a customer managed CMK

This section explains how to configure customer managed CMK for
GKE on AWS components. Before upgrading existing components, see
[Using CMK with existing clusters](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/using-cmk.md.txt#upgrade-existing).

### Bastion host and management service

You configure a customer managed CMK for your bastion host root volume,
management service root volume, and management service etcd data volumes in your
`anthos-gke.yaml` before setting up the management service. For more
information, see,
[Integrating existing infrastructure](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/integrating-existing-infrastructure.md.txt)
and the
[AWSManagementService](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/reference/awsmanagementservice.md.txt#rootvolume)
reference.

### User cluster control plane volumes

You configure a customer managed CMK for your user cluster control plane and
etcd data volumes in your [AWSCluster](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/reference/awscluster.md.txt)
definition. For more information, see
[Creating a custom user cluster](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/creating-custom-user-cluster.md.txt).

### User cluster node root volumes

You configure a customer managed CMK on user cluster node root volumes in your
[AWSNodePool](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/reference/awsnodepools.md.txt)
definition. For more information, see
[Creating a custom user cluster](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/creating-custom-user-cluster.md.txt).

### Using CMK with existing clusters

> [!CAUTION]
> **Caution:** If you enable a customer managed CMK on an existing management service or user cluster control plane *etcd* data volume, you will lose cluster configuration. To encrypt etcd data volumes, you must create a new management service or user clusters.

You can add customer managed CMK configuration to the following existing
components:

- Bastion host root volume
- Management service root volumes
- User cluster control plane root volumes
- User cluster node root volumes

GKE on AWS recreates these volumes after an upgrade or configuration
change. To add customer managed CMK configuration to existing components,
follow the instructions in
[Upgrading GKE on AWS](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/upgrading.md.txt). When you
modify your resource configuration, modify the following fields:

| Resource | Field |
|---|---|
| AWSManagementService | [`spec.rootVolume and spec.bastionHost.rootVolume`](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/reference/awsmanagementservice.md.txt#rootvolume) |
| AWSCluster | [`spec.controlPlane.rootVolume`](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/reference/awscluster.md.txt#speccontrolplanerootvolume_and_specetcdrootvolume) |
| AWSNodePools | [`spec.rootVolume`](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/reference/awsnodepools.md.txt#specrootvolume) |

When you complete the upgrade, GKE on AWS creates new resources with
the customer managed CMK volumes attached.

## What's next

- [Create a custom user cluster](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/creating-custom-user-cluster.md.txt).
- [Upgrade GKE on AWS](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/upgrading.md.txt)