Authenticate workloads to other workloads over mTLS

This document describes how to set up automatic provisioning and lifecycle management of managed workload identities for Compute Engine. You configure CA pools to issue certificates using Certificate Authority Service (CA), which is a highly-available, scalable Google Cloud service that simplifies and automates the deployment, management, and security of CA services. Each VM is provisioned with X.509 credentials from the configured CA pool. These credentials can then be used to establish mTLS connections.

Before you begin

  • Review the Managed workload identities overview documentation.

  • Enable the Compute Engine API:

    Roles required to enable APIs

    To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains the serviceusage.services.enable permission. Learn how to grant roles. 

    gcloud services enable compute.googleapis.com

Required roles

To get the permissions that you need to create VMs that use managed workload identity certificates for authentication to other workloads, ask your administrator to grant you the following IAM roles on the project:

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Overview

With managed workload identities for Compute Engine, you can implement mutually authenticated and encrypted communications between any two Compute Engine VMs. Workload applications running on the configured VMs can use the X.509 credentials for per-VM mTLS. The guest agent automatically rotates these mTLS certificates and refreshes credentials on the VM.

By using managed workload identities, you can centrally administer identity for your workloads. Compute Engine VMs can be authorized to receive a workload identity that is based on key attributes, such as the instance ID, instance name, or the attached service account. For the workloads to be attested, the compute administrator or developer must create VM instances for the workload with a matching instance name, instance ID, service account name, or service account ID.

Managed workload identities conforms to the open source Secure Production Identity Framework for Everyone (SPIFFE) standard. SPIFFE IDs for Google Cloud workloads are in the following format: spiffe://<TRUST_DOMAIN/ns/NAMESPACE/sa/WORKLOAD_ID. The TRUST_DOMAIN corresponds to an Identity and Access Management (IAM) workload identity pool.

Managed workload identities are defined within a workload identity pool, which acts as a top-level container to hold workload identities that have a common trust boundary. Under a workload identity pool, identities are organized into administrative boundaries called namespaces. For a workload to get a managed identity, you must define attestation policies to authorize it. A workload attestation policy ensures that only trusted workloads can assert a managed identity.

To use managed workload identities for your applications, you must perform the following tasks:

  1. Security Administrator:

  2. Compute Administrator:

Configure managed workload identities in Identity and Access Management

  • Follow the instructions in Configure managed workload identities authentication .

    These instructions detail how to complete the following:

    • Create a workload identity pool.
    • Create namespaces in the workload identity pool. You use the namespaces to create administrative boundaries for your managed workload identities, for example, a namespace for each of the applications owned by your organization.
    • Create a managed workload identity in a namespace in the workload identity pool. For example, you might create a namespace for an application and create managed identities within that namespace for the microservices that support that application.
    • Optional: Create a service account to create attestation rules based on service account ID.
    • Create a workload attestation policy that allows your workload to be issued credentials for the managed workload identity. Compute Engine VMs can be authorized to receive a managed workload identity based on key attributes such as the instance ID, instance name, or the attached service account.
    • Configure Certificate Authority Service to issue certificates for managed workload identities:
      • Configure the root CA pool
      • Configure the subordinate CAs
      • Authorize the managed workload identity pool to request certificates from the CA pool

Enable managed workload identities for individual compute instances

You can enable managed workload identities for a compute instance either when creating the instance or by updating the configuration for an existing instance.

Create instances with managed workload identities enabled

When you create a compute instance, to enable the managed workload identities feature for the instance, you must do the following:

  • If the attestation of the instance for the workload identity is based on a service account, then specify that service account

  • Include the --identity, --identity-type and --identity-certificate flags

gcloud

Use the gcloud alpha compute instances create command to create a new instance.

  1. If you're using a service account for attestation, include the service account information when creating the instance.

    You can add additional lines to the command to configure the instance, such as the machine type and image, instead of using the default values. For more information, see Create and start a VM instance.

    gcloud alpha compute instances create INSTANCE_NAME \
   --zone=INSTANCE_ZONE \
   --service-account SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com \
   --identity=//TRUST_DOMAIN/ns/NAMESPACE/sa/WORKLOAD_IDENTIFIER \
   --identity-type=IDENTITY_TYPE \
   --identity-certificate \
...

    Replace the following:

    • INSTANCE_NAME: a unique name for the instance. For details on valid instance names, see Name resources.
    • INSTANCE_ZONE: the zone to create the instance in.
    • SERVICE_ACCOUNT_NAME: the name of the service account that is allowed to receive the managed workload identity.
    • PROJECT_ID: the ID of the project where the service account was created.
    • TRUST_DOMAIN: the ID for the trust domain, in the following format: 
        $POOL_ID.global.$PROJECT_NUMBER.workload.id.goog

      The trust domain value uses the following placeholders:

      • POOL_ID: the name of the workload identity pool
      • PROJECT_NUMBER: the project number for the project in which the workload identity pool was created
    • NAMESPACE: the name of the namespace that contains the workload attestation policy that authorizes the workload to assert a managed workload identity
    • WORKLOAD_IDENTIFIER: the name of the managed workload identity that was created in the namespace of the workload identity pool
    • IDENTITY_TYPE: the format of the identity, with a value of either `SERVICE_ACCOUNT` or `WORKLOAD_IDENTITY`

Enable managed workload identities on existing instances

To enable managed workload identities for an existing compute instance, update the instance to configure the following:

  • If the attestation of the instance for the workload identity is based on a service account and the instance doesn't already have an attached service account, then create and attach a service account to the instance.
  • Enable the feature using the --identity-certificate flag and specify which managed workload identity to use with the --identity flag.

  • Restart the VM.

gcloud

Use the gcloud alpha compute instances update command to update the configuration for an instance.

  1. If the instance doesn't already have an attached service account, attach the service account to the instance.

  2. Update the configuration for an existing instance to enable managed workload identities.

    gcloud alpha compute instances update INSTANCE_NAME \
   --zone=ZONE \
   --identity-certificate \
   --identity=//TRUST_DOMAIN/ns/NAMESPACE/sa/WORKLOAD_IDENTIFIER

    Replace the following:

    • INSTANCE_NAME: the instance name
    • ZONE: the zone where the instance is located
    • TRUST_DOMAIN: the ID for the trust domain, in the following format: none $POOL_ID.global.$PROJECT_NUMBER.workload.id.goog

    The trust domain value uses the following placeholders: * POOL_ID: the name of the workload identity pool * PROJECT_NUMBER: the project number for the project in which the workload identity pool was created + NAMESPACE: the name of the namespace that contains the workload attestation policy that authorizes the workload to assert a managed workload identity + WORKLOAD_IDENTIFIER: the name of the managed workload identity that was created in the namespace of the workload identity pool

  3. Stop the VM.

    gcloud alpha compute instances stop INSTANCE_NAME \
   --zone=ZONE

    Replace the following:

    • INSTANCE_NAME: the instance name
    • ZONE: the zone where the instance is located

  4. Start the VM.

    gcloud alpha compute instances start INSTANCE_NAME \
   --zone=ZONE

    Replace the following:

    • INSTANCE_NAME: the instance name
    • ZONE: the zone where the instance is located

Access workload credentials on a Linux VM

After you create and start a compute instance that uses managed workload identities, the Google Cloud infrastructure (including the Guest Agent) automatically provisions and manages the rotation of the credentials for the instance.

The workload certificates, including the private key and trust bundles, are placed into a specific directory on the VM's file system: /var/run/secrets/workload-spiffe-credentials. Applications can read the certificates, private key and the trust bundle from the file system directly to establish mTLS connections.

The Compute Engine guest agent on the guest OS runs a workload refresh job and, if needed, creates the directory /var/run/secrets/workload-spiffe-credentials. The directory permissions are set to 0755 (rwxr-xr-x).

The directory contains the following files created with 0644 (rw-r--r--) permissions:

  • private_key.pem: a PEM-formatted private key
  • certificates.pem: a bundle of PEM-formatted X.509 certificates that can be presented to other VMs as the client certificate chain, or used as a server certificate chain.

  • ca_certificates.pem: a bundle of PEM-formatted X.509 certificates to use as trust anchors when validating the certificates of peers.

    spiffe://POOL_ID.global.PROJECT_NUMBER.workload.id.goog

  • config_status: a log file containing error messages.

Refreshing the credentials and trust bundle

The Compute Engine control plane automatically rotates the managed workload identity credentials and the trust anchors periodically.

If your applications use the file system to access the workload credentials and trust bundle, the Compute Engine guest agent automatically refreshes the credentials and the trust bundle, for example, when the mTLS certificates are rotated.

Update certificate issuance and trust config

You can modify the certificate issuance config and trust config for a VM that uses managed workload identities.

Update individual Compute Engine VMs

To update the certificate issuance config and trust config, you update the workload identity pool. Use the gcloud alpha iam workload-identity-pools update command, as shown in the following example:

gcloud alpha iam workload-identity-pools update WORKLOAD_IDENTITY_POOL_ID \
    --location="global" \
    --inline-certificate-issuance-config-file=CIC_JSON_FILE_PATH \
    --inline-trust-config-file=TC_JSON_FILE_PATH \
    --project=PROJECT_ID

Replace the following:

  • WORKLOAD_IDENTITY_POOL_ID: The ID of the workload identity pool.
  • CIC_JSON_FILE_PATH: The path to the new JSON-formatted certificate issuance config file (cic.json).
  • TC_JSON_FILE_PATH: Optional: The path to the new JSON-formatted trust config file (tc.json). If your workloads authenticate across different trust domains, you must specify this file. Otherwise you can omit the --inline-trust-config setting.
  • PROJECT_ID: Your Google Cloud project name.

Troubleshoot

To find methods for diagnosing and resolving common errors related to retrieving workload credentials, refer to the Troubleshoot workload to workload authentication documentation.

To troubleshoot issues with misconfigured pools or attestation policies, see the audit logs for workload identity pools.

What's next