<br />

<br />


[Managed Airflow (Gen 3)](https://docs.cloud.google.com/composer/docs/composer-3/configure-cmek-encryption.md.txt "View this page for Managed Airflow (Gen 3)") \| [Managed Airflow (Gen 2)](https://docs.cloud.google.com/composer/docs/composer-2/configure-cmek-encryption.md.txt "View this page for Managed Airflow (Gen 2)") \| **Managed Airflow (Legacy Gen 1)**

<br />

<br />

<br />

<br />

<br />

<br />

This page describes how to use
[Customer Managed Encryption Keys (CMEK)](https://docs.cloud.google.com/kms/docs/cmek.md.txt) to protect
Managed Airflow environments.

### Data protected with CMEK encryption

Managed Airflow protects the following data with CMEK encryption:

- Contents and schema of the Airflow database
- Environment and Airflow task logs in Cloud Logging
- Contents of the environment's bucket
- Secrets stored in the environment's cluster
- Persistent disks used by the task queue
- Container images of environment components stored in Artifact Registry repositories

For specific details about how the data is encrypted, see:

- [About customer-managed encryption keys (CMEK)](https://docs.cloud.google.com/sql/docs/postgres/cmek.md.txt) in the Cloud SQL documentation.
- [Use customer-managed encryption keys (CMEK)](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/using-cmek.md.txt) and [Encrypt secrets at the application layer](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/encrypting-secrets.md.txt) in the Google Kubernetes Engine documentation.
- [Customer-managed encryption keys](https://docs.cloud.google.com/storage/docs/encryption/customer-managed-keys.md.txt) in the Cloud Storage documentation.
- [Configure CMEK for Cloud Logging](https://docs.cloud.google.com/logging/docs/routing/managed-encryption.md.txt) in the Cloud Logging documentation.

### Data not protected with CMEK encryption

Cloud Monitoring does not support CMEK encryption. The name of your
environment and names of DAGs are stored in the Monitoring
database in the encrypted form using Google-owned and Google-managed encryption keys.

Managed Airflow stores the following information protected with
Google-owned and Google-managed encryption keys, not customer-managed keys:

- Environment name
- Airflow configuration overrides
- Environment variables

- Descriptions of allowed IP ranges

- IP ranges

- Labels

- The names of some parameters stored by Managed Airflow can include
  a sub-string of environment's name.

### Rotation of CMEK keys for Managed Airflow

After you configure encryption in your environment using CMEK keys, you
might also want to consider rotating these keys on a regular basis as described
in [KMS documentation](https://docs.cloud.google.com/kms/docs/key-rotation.md.txt).

When you rotate a CMEK key, data encrypted with previous key versions is
**not automatically re-encrypted** with the new key version.
For more information, see [Re-encrypting data](https://docs.cloud.google.com/kms/docs/re-encrypt-data.md.txt).

Specifically, this applies to:

- Objects stored in the environment's bucket.
- Data stored in the Airflow database.
- All [other data objects](https://docs.cloud.google.com/composer/docs/composer-1/configure-cmek-encryption.md.txt#about-protected-data) encrypted with CMEK in the Managed Airflow environment.
- Container images of environment components stored in Artifact Registry repositories.

## Use a customer-managed encryption key for your environment

> [!WARNING]
> **Warning:** If you create an environment with a customer-managed encryption key and later disable or delete the key, the environment becomes **unusable** . It is **not possible to recover** an environment if its encryption key is deleted or disabled.

### Before you begin

- You can configure CMEK only when you create an environment. It is not
  possible to enable CMEK for an existing environment.

- Managed Airflow supports CMEK encryption using
  [keys stored in External Key Managers](https://docs.cloud.google.com/kms/docs/ekm.md.txt).

- You must create a CMEK key in the same region where your environments
  are located. You cannot use multi-regional or global keys.

  <br />

- If your environment runs inside a [VPC Service Controls](https://docs.cloud.google.com/composer/docs/composer-1/configure-vpc-sc.md.txt)
  perimeter, add the Cloud Key Management Service API and the Artifact Registry API to the perimeter.

- If your environment runs inside a [VPC Service Controls](https://docs.cloud.google.com/composer/docs/composer-1/configure-vpc-sc.md.txt)
  perimeter, the project that hosts the Cloud Key Management Service keys must
  [belong to the same VPC Service Controls perimeter as the environment](https://docs.cloud.google.com/composer/docs/composer-1/configure-vpc-sc.md.txt#shared-vpc-cmek-perimeters).

- Enable the Artifact Registry API.

  ### Console


  Enable the Artifact Registry API, if it is not already enabled.


  **Roles required to enable APIs**


  To enable APIs, you need the `serviceusage.services.enable` permission. If you
  created the project, then you likely already have this permission through the
  Owner role (`roles/owner`). Otherwise, you can get this permission through the
  Service Usage Admin role (`roles/serviceusage.serviceUsageAdmin`).
  [Learn how to grant roles](https://docs.cloud.google.com/iam/docs/granting-changing-revoking-access.md.txt).

  [Enable the API](https://console.cloud.google.com/apis/enableflow?apiid=artifactregistry.googleapis.com)

  <br />

  ### gcloud


  Enable the Artifact Registry API, if it is not already enabled:


  **Roles required to enable APIs**


  To enable APIs, you need the `serviceusage.services.enable` permission. If you
  created the project, then you likely already have this permission through the
  Owner role (`roles/owner`). Otherwise, you can get this permission through the
  Service Usage Admin role (`roles/serviceusage.serviceUsageAdmin`).
  [Learn how to grant roles](https://docs.cloud.google.com/iam/docs/granting-changing-revoking-access.md.txt).

  ```bash
  gcloud services enable artifactregistry.googleapis.com
  ```

  <br />

### Step 1. Create a customer-managed encryption key

Follow the steps outlined in
[Creating symmetric encryption keys](https://docs.cloud.google.com/kms/docs/creating-keys) to create a key in the
region where your environment is located.

### Step 2. Grant roles to service agents

> [!IMPORTANT]
> **Important:** If your project has a [Domain restricted sharing](https://docs.cloud.google.com/resource-manager/docs/organization-policy/restricting-domains) policy that excludes addresses from the google.com domain. You must turn off (or expand) this policy to grant the required roles to service agents. After you grant the roles, you can revert the policy changes. This policy does not affect the creation of environments.

### Console

Skip this step. You grant permissions to
[service agents](https://docs.cloud.google.com/iam/docs/service-agents.md.txt) when you
[specify a key for your environment](https://docs.cloud.google.com/composer/docs/composer-1/configure-cmek-encryption.md.txt#create-environment).

### gcloud

The following [service agents](https://docs.cloud.google.com/iam/docs/service-agents.md.txt) must have
the **Cloud KMS CryptoKey Encrypter/Decrypter** role on the key that you
use for your environment.

Replace `PROJECT_NUMBER` with your
[project's number](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects.md.txt).

| Service agent name | Service account email | API service name |
|---|---|---|
| Managed Airflow Service Agent | service-<var class="edit" translate="no">PROJECT_NUMBER</var>@cloudcomposer-accounts.iam.gserviceaccount.com | composer.googleapis.com |
| Artifact Registry Service Agent | service-<var class="edit" translate="no">PROJECT_NUMBER</var>@gcp-sa-artifactregistry.iam.gserviceaccount.com | artifactregistry.googleapis.com |
| Kubernetes Engine Service Agent | service-<var class="edit" translate="no">PROJECT_NUMBER</var>@container-engine-robot.iam.gserviceaccount.com | container.googleapis.com |
| Pub/Sub Service Agent | service-<var class="edit" translate="no">PROJECT_NUMBER</var>@gcp-sa-pubsub.iam.gserviceaccount.com | pubsub.googleapis.com |
| Compute Engine Service Agent | service-<var class="edit" translate="no">PROJECT_NUMBER</var>@compute-system.iam.gserviceaccount.com | compute.googleapis.com |
| Cloud Storage Service Agent | service-<var class="edit" translate="no">PROJECT_NUMBER</var>@gs-project-accounts.iam.gserviceaccount.com | Grant encrypt/decrypt permissions with `gcloud storage service-agent --authorize-cmek` |

1. (If required) If some of these service accounts are not present in your
   project, it means that an identity for this service is not yet created.
   This can happen, for example, if you did not yet create any
   Managed Airflow environments in your project.

   To add these service accounts, create identities for the listed services
   with the following command:

   ```bash
   gcloud beta services identity create \
     --service=API_SERVICE_NAME
   ```

   Replace `API_SERVICE_NAME` with the
   [API service name](https://docs.cloud.google.com/apis/design/glossary#api_service_name) of a service
   that does not have a service account in your project.

   Example:

   ```bash
   gcloud beta services identity create \
     --service=composer.googleapis.com
   ```
2. Grant permissions to service agents:

   1. Grant the role to Managed Airflow Service Agent:

      ```bash
      gcloud kms keys add-iam-policy-binding KEY_NAME \
        --location KEY_LOCATION \
        --keyring KEY_RING_NAME \
        --member=serviceAccount:service-PROJECT_NUMBER@cloudcomposer-accounts.iam.gserviceaccount.com \
        --role=roles/cloudkms.cryptoKeyEncrypterDecrypter \
        --project KEY_PROJECT_ID
      ```
   2. Grant the role to Artifact Registry Service Agent:

      ```bash
      gcloud kms keys add-iam-policy-binding KEY_NAME \
        --location KEY_LOCATION \
        --keyring KEY_RING_NAME \
        --member=serviceAccount:service-PROJECT_NUMBER@gcp-sa-artifactregistry.iam.gserviceaccount.com \
        --role=roles/cloudkms.cryptoKeyEncrypterDecrypter \
        --project KEY_PROJECT_ID
      ```
   3. Grant the role to GKE Service Agent:

      ```bash
      gcloud kms keys add-iam-policy-binding KEY_NAME \
        --location KEY_LOCATION \
        --keyring KEY_RING_NAME \
        --member=serviceAccount:service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com \
        --role=roles/cloudkms.cryptoKeyEncrypterDecrypter \
        --project KEY_PROJECT_ID
      ```
   4. Grant the role to Pub/Sub Service Agent:

      ```bash
      gcloud kms keys add-iam-policy-binding KEY_NAME \
        --location KEY_LOCATION \
        --keyring KEY_RING_NAME \
        --member=serviceAccount:service-PROJECT_NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com \
        --role=roles/cloudkms.cryptoKeyEncrypterDecrypter \
        --project KEY_PROJECT_ID
      ```
   5. Grant the role to Compute Engine Service Agent:

      ```bash
      gcloud kms keys add-iam-policy-binding KEY_NAME \
        --location KEY_LOCATION \
        --keyring KEY_RING_NAME \
        --member=serviceAccount:service-PROJECT_NUMBER@compute-system.iam.gserviceaccount.com \
        --role=roles/cloudkms.cryptoKeyEncrypterDecrypter \
        --project KEY_PROJECT_ID
      ```
   6. Grant encrypt/decrypt permissions to
      Cloud Storage Service Agent.

      ```bash
      gcloud storage service-agent \
        --authorize-cmek=projects/KEY_PROJECT_ID/locations/KEY_LOCATION/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME
      ```

   Replace:
   - `PROJECT_ID` with your [project's ID](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects.md.txt).
   - `KEY_PROJECT_ID` with the ID of the project that stores your customer-managed key. If you use a key from another project, this value is different from your project's ID. If you use a key from the same project, this value is your project's ID.
   - `PROJECT_NUMBER` with your [project's number](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects.md.txt).
   - `KEY_LOCATION` with your customer-managed key location. This location must be the same as your environment's location.
   - `KEY_NAME` with your customer-managed key name.
   - `KEY_RING_NAME` with the keyring that stores your customer-managed key.

   To get these values, you can run `gcloud projects describe`,
   `gcloud kms keyrings list`, and `gcloud kms keys describe` commands.

### Step 3. Create an environment with CMEK

After you [create a customer-managed encryption key](https://docs.cloud.google.com/composer/docs/composer-1/configure-cmek-encryption.md.txt#create-key), you can use
it to create Managed Airflow environments.

### Console

When you [create an environment](https://docs.cloud.google.com/composer/docs/composer-1/create-environments.md.txt):

1.

   Expand the **Networking, Airflow config overrides, and additional features**
   section.

   In the **Data encryption** section, select
   **Cloud Key Management Service key**.

2. In the **Select a Cloud Key Management Service key** drop-down list, select your key.

3. If additional setup is required, a message appears to inform you. In
   this case:

   1. Click **Open wizard**.

   2. In the **Prepare CMEK key for usage in Managed Airflow**
      dialog, view the list of [service agents](https://docs.cloud.google.com/iam/docs/service-agents.md.txt)
      that must have the **Cloud KMS CryptoKey Encrypter/Decrypter** role
      on the key.

   3. To grant the required roles and permissions, click **Grant**.

   > [!IMPORTANT]
   > **Important:** If your project has a [Domain restricted sharing](https://docs.cloud.google.com/resource-manager/docs/organization-policy/restricting-domains) policy that excludes addresses from the google.com domain. You must turn off (or expand) this policy to grant the required roles to service agents. After you grant the roles, you can revert the policy changes. This policy does not affect the creation of environments.

### gcloud

The `--kms-key` argument specifies a customer-managed enryption key for your
environment.

For more information about creating environments, see
[Create environments](https://docs.cloud.google.com/composer/docs/composer-1/create-environments.md.txt). For example, you might want to specify
other parameters for your environment.

```bash
gcloud composer environments create ENVIRONMENT_NAME \
  --location LOCATION \
  --image-version IMAGE_VERSION \
  --kms-key projects/KEY_PROJECT_ID/locations/KEY_LOCATION/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME
```

Replace:

- `ENVIRONMENT_NAME` with the name of the environment.
- `IMAGE_VERSION` with the name of the Managed Airflow image.
- `KEY_PROJECT_ID` with the ID of the project where the key is located. If you use a key from another project, this value is different from your project's ID. If you use a key from the same project, this value is your project's ID.
- `LOCATION` with the region where the environment is located.
- `KEY_LOCATION` with the location of your customer-managed key. This location must be the same as your environment's location.
- `KEY_NAME` with your customer-managed key name.
- `KEY_RING_NAME` with the keyring that stores your customer-managed key.

Example:

```bash
gcloud composer environments create example-environment \
  --location us-central1 \
  --image-version composer-1.20.12-airflow-1.10.15 \
  --kms-key projects/example-project/locations/us-central1/keyRings/example-key-ring/cryptoKeys/example-key
```

## View the environment's encryption configuration

You can view encryption configuration for an existing environment:

### Console

1. In Google Cloud console, go to the **Environments** page.

   [Go to Environments](https://console.cloud.google.com/composer/environments)
2. In the list of environments, click the name of your environment.
   The **Environment details** page opens.

3. Go to the **Environment configuration** tab.

4. Details about encryption are listed in the **Data encryption key** item.

### gcloud

Run the following `gcloud` command to view the encryption configuration

```bash
gcloud composer environments describe \
  ENVIRONMENT_NAME \
  --location LOCATION \
  --format="value(config.encryptionConfig)"
```

Replace:

- `ENVIRONMENT_NAME` with the name of the environment.
- `LOCATION` with the region where the environment is located.

Example:

```bash
gcloud composer environments describe \
  example-environment \
  --location us-central1 \
  --format="value(config.encryptionConfig)"
```

## Use CMEK for Managed Airflow logs

Cloud Logging supports encrypting Log storage with CMEK keys. We recommend
to use the standard CMEK procedure for encrypting logs with CMEK keys.

To encrypt logs with CMEK keys, follow the instructions outlined in
[Manage the keys that protect Logging storage data](https://docs.cloud.google.com/logging/docs/routing/managed-encryption-storage.md.txt).

> [!WARNING]
> **Deprecated:** The following instructions about redirecting Managed Airflow logs to a CMEK-encrypted Cloud Storage bucket are deprecated. If you use this approach in your project, consider switching to encrypting Log storage with CMEK keys instead.

### Redirect Managed Airflow logs to a CMEK-encrypted Cloud Storage bucket

If you expect your logs to contain sensitive data, you might want to
redirect Managed Airflow logs to a CMEK-encrypted
Cloud Storage bucket by using
[Log Router](https://docs.cloud.google.com/logging/docs/routing/overview.md.txt#log-router). Doing so prevents your
logs from being sent to Monitoring.

If you require support from Cloud Customer Care, you might need to grant Google
support engineers access to the Managed Airflow logs stored in
Cloud Storage.

> [!NOTE]
> **Note:** [Audit logs](https://docs.cloud.google.com/logging/docs/audit.md.txt) cannot be excluded. They are always sent to the default storage.

### gcloud

1. Create a new Cloud Storage bucket to store the logs.

   ```bash
   gcloud storage buckets create gs://BUCKET_NAME --location=LOCATION
   ```

   Replace:
   - `LOCATION` with the region where the environment is located.
   - `BUCKET_NAME` with the name for the bucket.

   Example:

   ```bash
   gcloud storage buckets create gs://composer-logs-us-central1-example-environment --location=us-central1
   ```
2. Encrypt the bucket with your CMEK key.

   ```bash
   gcloud storage buckets update gs://BUCKET_NAME \
     --default-encryption-key=projects/KEY_PROJECT_ID/locations/KEY_LOCATION/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME
   ```

   Replace:
   - `KEY_PROJECT_ID` with the ID of the project where the key is located. If you use a key from another project, this value is different from your project's ID. If you use a key from the same project, this value is your project's ID.
   - `KEY_LOCATION` with the location of your customer-managed key. This location must be the same as your environment's location.
   - `KEY_RING_NAME` with the keyring that stores your customer-managed key.
   - `KEY_NAME` with your customer-managed key name.
   - `BUCKET_NAME` with the name of the bucket.

   Example:

   ```bash
   gcloud storage buckets update gs://composer-logs-us-central1-example-environment \
     --default-encryption-key=projects/example-project/locations/us-central1/keyRings/example-key-ring/cryptoKeys/example-key
   ```
3. Create a new log sink.

   ```bash
   gcloud logging sinks create \
   composer-log-sink-ENVIRONMENT_NAME \
   storage.googleapis.com/BUCKET_NAME \
   --log-filter "resource.type=cloud_composer_environment AND resource.labels.environment_name=ENVIRONMENT_NAME AND resource.labels.location=LOCATION"
   ```

   Replace:
   - `ENVIRONMENT_NAME` with the name of the environment.
   - `LOCATION` with the region where the environment is located.
   - `BUCKET_NAME` with the name of the bucket.

   Example:

   ```bash
   gcloud logging sinks create \
   composer-log-sink-example-environment \
   storage.googleapis.com/composer-logs-us-central1-example-environment \
   --log-filter "resource.type=cloud_composer_environment AND resource.labels.environment_name=example-environment AND resource.labels.location=us-central1"
   ```
4. Grant the **Storage Object Creator** role to the service account for this
   bucket. The service account is shown in the result of the previous
   command).

   ```bash
   gcloud projects add-iam-policy-binding \
     PROJECT_ID \
     --member="serviceAccount:LOGGING_SERVICE_AGENT" \
     --role="roles/storage.objectCreator" \
     --condition=None
   ```

   Replace:
   - `PROJECT_ID` with the [Project ID](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects.md.txt).
   - `LOGGING_SERVICE_AGENT` with the Logging service agent account for this bucket. The name of this account is obtained on the previous step.

   Example:

   ```bash
   gcloud projects add-iam-policy-binding \
     example-project \
     --member="serviceAccount:example-sa@gcp-sa-logging.iam.gserviceaccount.com" \
     --role="roles/storage.objectCreator" \
     --condition=None
   ```
5. Exclude the logs for your new environment from
   Monitoring.

   ```bash
   gcloud beta logging sinks update _Default \
     --add-exclusion name=ENVIRONMENT_NAME-exclusion,filter="resource.type=cloud_composer_environment AND resource.labels.environment_name=ENVIRONMENT_NAME AND resource.labels.location=LOCATION"
   ```

   Replace:
   - `ENVIRONMENT_NAME` with the name of the environment.
   - `LOCATION` with the region where the environment is located.

   Example:

   ```bash
   gcloud beta logging sinks update _Default \
     --add-exclusion name=example-environment-exclusion,filter="resource.type=cloud_composer_environment AND resource.labels.environment_name=example-environment AND resource.labels.location=us-central1"
   ```
6. Add organization-level CMEK encryption to the Log Router.

   ```bash
   gcloud logging cmek-settings describe \
     --organization=ORGANIZATION_ID
   ```

   ```bash
   gcloud kms keys add-iam-policy-binding \
         --project=KEY_PROJECT_ID \
         --member LOGGING_SERVICE_AGENT \
         --role roles/cloudkms.cryptoKeyEncrypterDecrypter \
         --location=KEY_LOCATION \
         --keyring=KEY_RING_NAME \
         KEY_NAME
   ```

   ```bash
   gcloud logging cmek-settings update \
     --organization=ORGANIZATION_ID \
     --kms-project=KEY_PROJECT_ID \
     --kms-keyring=KEY_RING_NAME \
     --kms-location=KEY_LOCATION \
     --kms-key-name=KEY_NAME
   ```

   Replace:
   - `ORGANIZATION_ID` with your [organization ID](https://docs.cloud.google.com/resource-manager/docs/creating-managing-organization.md.txt#retrieving_your_organization_id).
   - `KEY_PROJECT_ID` with the ID of the project where the key is located. If you use a key from another project, this value is different from your project's ID. If you use a key from the same project, this value is your project's ID.
   - `KEY_RING_NAME` with the keyring that stores your customer-managed key.
   - `KEY_LOCATION` with the location of your customer-managed key. This location must be the same as your environment's location.
   - `KEY_NAME` with your customer-managed key name.

## What's next

- [Security overview](https://docs.cloud.google.com/composer/docs/composer-1/composer-security-overview.md.txt)
- [Create environments](https://docs.cloud.google.com/composer/docs/composer-1/create-environments.md.txt)