This document describes how to mitigate the impact of an attacker compromising
the [OAuth bearer tokens](https://docs.cloud.google.com/docs/authentication/token-types#access-tokens) that are used by the [gcloud CLI](https://docs.cloud.google.com/sdk/gcloud).

An attacker can compromise these OAuth tokens if they gain access to an endpoint
where a legitimate user account or service account has already authenticated
with the gcloud CLI. The attacker can then copy these tokens to another endpoint
that they control to make requests that impersonate the legitimate identity.
Even after you remove the attacker's access to the compromised endpoint, the
attacker can
continue to make authenticated API requests using the copied tokens. To help
mitigate this risk, you can control access to your systems by using
credentials that are short-lived and context-aware.

This document is intended for security teams or cloud architects who are
responsible
for securing their cloud resources from illegitimate access. This document
introduces the available controls that you can use to proactively reduce the
impact of compromised gcloud CLI OAuth tokens and remediate your
environment after an endpoint has been compromised.

## Overview

To understand how this threat works, you must understand how the
gcloud CLI stores OAuth 2.0 credentials and how those credentials
can be abused if compromised by an attacker.

### Types of credentials stored by the gcloud CLI

The gcloud CLI uses
[OAuth 2.0 access tokens](https://oauth.net/2/access-tokens/)
to authenticate requests for Google Cloud APIs. The OAuth flow varies by the
credential types used, but generally the access token and other credentials are
accessible locally. In each case, the access token expires after 60 minutes, but
other credential types might be persistent.

When you
[authorize the gcloud CLI with a user account](https://docs.cloud.google.com/sdk/docs/authenticate#user-accounts),
the gcloud CLI initiates a three-legged OAuth consent flow to access
Google Cloud APIs on the user's behalf. After the user completes the
consent flow, the gcloud CLI receives an access token and a refresh
token that allows it to request new access tokens. The long-lived refresh token persists until its
[expiration conditions are met](https://developers.google.com/identity/protocols/oauth2?_ga=2.56060843.1825758564.1654790150-690286853.1654790150#expiration).

When you
[authorize the gcloud CLI with a service account](https://docs.cloud.google.com/sdk/docs/authenticate#service-accounts),
the gcloud CLI initiates a two-legged OAuth flow to access
Google Cloud APIs as the service account identity. After you activate a
service account from a private key file, this key is used to periodically request
an access token. The long-lived private key is stored in the gcloud CLI
configuration and remains valid until you
[delete the service account key](https://docs.cloud.google.com/iam/docs/keys-create-delete#deleting).

When you run the gcloud CLI inside a Google Cloud
environment, like Compute Engine or Cloud Shell, the application can
automatically
find credentials and
[authenticate as a service account](https://docs.cloud.google.com/docs/authentication/production#automatically).
For example, in Compute Engine, an application like the gcloud CLI
can query the metadata server for an access token. Google manages and
rotates the private signing key that is used to create the access token, and the
long-lived credentials aren't exposed to the application.

When you
[authenticate with workload identity federation](https://docs.cloud.google.com/iam/docs/workload-identity-federation),
applications authenticate based on a credential from an external identity
provider and receive a federated short-lived access token. For more information
on how to store and manage
the long-lived credentials used by the external identity provider, see
[best practices for using workload identity federation](https://docs.cloud.google.com/iam/docs/best-practices-for-using-workload-identity-federation).

### How an attacker can use compromised OAuth tokens

If an attacker manages to compromise an endpoint, credentials such as OAuth
tokens are valuable targets because they let attackers persist or escalate
their access.

A developer might have a legitimate need to view their own credentials when
writing and debugging code. For example, a developer might need to
[authenticate for using REST](https://docs.cloud.google.com/docs/authentication/rest)
requests to Google Cloud services when working with an unsupported client
library. The developer can view the credentials through various methods,
including the following:

- Viewing the [gcloud CLI configuration files](https://docs.cloud.google.com/sdk/docs/configurations) on the local filesystem
- Querying the [Compute Engine metadata server](https://docs.cloud.google.com/compute/docs/metadata/overview)
- Using commands like `gcloud auth print-access-token` or `gcloud
  auth describe IDENTITY`

However, an attacker might use these same techniques after they have
compromised an endpoint.

If an attacker compromises an endpoint, such as a developer workstation, the
primary threat is that the attacker can run gcloud CLI commands or
other code with the legitimate credentials of the authenticated identity. In
addition, the attacker might copy the OAuth tokens to another endpoint that they
control to persist their access. When this credential theft happens, there is a
secondary threat that the attacker can still use the long-lived OAuth tokens to
have persistent access even after you remove access to the compromised
endpoint.

If the attacker manages to compromise OAuth tokens, they can
complete the following actions:

- An attacker can impersonate the compromised user or service account. API traffic that uses the compromised tokens is logged as if it came from the compromised user or service account, making it difficult to distinguish normal and malicious activity in logs.
- An attacker can refresh the access token indefinitely using a persistent refresh token associated with a user or a private key associated with a service account.
- An attacker can bypass authentication with the user's password or 2-step verification because the tokens are granted after the sign-in flow.

## Best practices for mitigating risks

Implement the controls described in the following sections to help mitigate
the risk of compromised gcloud CLI tokens. If you're following the
security best practices that are described in the
[enterprise foundations blueprint](https://docs.cloud.google.com/architecture/blueprints/security-foundations/authentication-authorization#administrative-controls-for-cloud-identity)
or
[landing zone design in Google Cloud](https://docs.cloud.google.com/architecture/landing-zones/decide-security),
you might already have these controls in place.

### Set session length for Google Cloud services

To reduce how long an attacker can exploit a compromised token,
[set the session length for Google Cloud services](https://support.google.com/a/answer/9368756).
For new customers, a [default session length of 16 hours](https://cloud.google.com/blog/products/identity-security/improve-security-posture-with-time-bound-session-length)
is automatically enforced. Customers who created their Google Cloud organization
prior to 2023 might have a default setting to never require reauthentication. Review
this setting to ensure that you have a reauthentication policy with a session length
that is between 1 and 24 hours. The reauthentication policy invalidates the refresh
token and forces the user to regularly reauthenticate the gcloud CLI with
their password or security key.

The session length for Google Cloud services is a distinct
setting from
[session length for Google services](https://support.google.com/cloudidentity/answer/7576830),
which controls web sessions for sign-in across Google Workspace services but
doesn't control reauthentication for the Google Cloud. If you use
Google Workspace services, set the session length for both.

### Configure VPC Service Controls

Configure
[VPC Service Controls](https://docs.cloud.google.com/vpc-service-controls/docs/overview)
across your environment to help ensure that only Google Cloud API traffic that
originates within your defined perimeter can access
[supported resources](https://docs.cloud.google.com/vpc-service-controls/docs/supported-products).
The service perimeter limits the usefulness of compromised credentials because
the perimeter blocks requests to restricted services that originate from
attacker-controlled endpoints that are outside of your environment.

### Configure Chrome Enterprise Premium

Configure Chrome Enterprise Premium policies to help
[secure the Google Cloud console and Google Cloud APIs](https://docs.cloud.google.com/chrome-enterprise-premium/docs/securing-console-and-apis). Configure a Chrome Enterprise Premium access level
and binding to selectively allow attributes that are evaluated on every API
request, including IP-based access or [certificate-based access for mutual TLS](https://docs.cloud.google.com/chrome-enterprise-premium/docs/securing-resources-with-certificate-based-access).
Requests that use compromised authorization credentials but don't meet the
conditions that are defined in your Chrome Enterprise Premium policy are rejected.

> [!NOTE]
> **Note:** Not all [device attributes collected by endpoint verification](https://docs.cloud.google.com/endpoint-verification/docs/device-information) are evaluated during every API request.

Chrome Enterprise Premium is a
user-centric control that rejects user API traffic that doesn't meet defined
conditions. VPC Service Controls is a resource-centric control that defines
the perimeters within which resources can communicate. VPC Service Controls
applies to all user identities and service account identities, but
Chrome Enterprise Premium applies only to user identities within your organization.
When used together, Chrome Enterprise Premium and VPC Service Controls reduce the
effectiveness of compromised credentials on an attacker-controlled machine that
is outside of your environment.

### Enforce 2-step verification for remote server access

If you let developers access Compute Engine resources using SSH,
configure
[OS Login with 2-step verification](https://docs.cloud.google.com/compute/docs/oslogin/setup-two-factor-authentication).
This enforces an additional checkpoint where a user must reauthenticate with
their password or security key. An attacker with compromised OAuth tokens but no
password or security key is blocked by this feature.

Remote Desktop Protocol (RDP) access to Windows instances on Compute Engine doesn't support the
OS Login service, so 2-step verification can't be granularly enforced for RDP
sessions. When using
[IAP Desktop](https://docs.cloud.google.com/compute/docs/instances/connecting-to-windows#iap-desktop)
or Google Chrome-based RDP plugins, set coarse-grained controls like
[session length for Google services](https://support.google.com/a/answer/7576830)
and
[2-step verification](https://support.google.com/a/answer/9176657)
settings for the user's web sessions and disable the **Allow user to trust the
device** setting under 2-step verification.

### Restrict the use of service account keys

When you use a service account key to authenticate, the key value is stored in
the gcloud CLI configuration files, separately from the downloaded
key file. An attacker with access to your environment could copy the key from
the gcloud CLI configuration or copy the key file from your local
filesystem or internal code repository. Therefore, in addition to your plan to
mitigate compromised access tokens, consider how you manage downloaded service
account key files.

Review
[more secure alternatives for authentication](https://docs.cloud.google.com/docs/authentication#auth-decision-tree)
to reduce or eliminate your use cases that depend on a service account key, and
enforce the organization policy constraint [iam.disableServiceAccountKeyCreation](https://docs.cloud.google.com/resource-manager/docs/organization-policy/restricting-service-accounts#disable_service_account_key_creation)
to disable service account key creation.

### Consider the principle of least privilege

When designing IAM policies, consider
[least privilege](https://docs.cloud.google.com/iam/docs/using-iam-securely#least_privilege).
Grant users the roles that they require to accomplish a task at the smallest
scope. Don't grant them roles that they don't require.
[Review and apply role recommendations](https://docs.cloud.google.com/policy-intelligence/docs/review-apply-role-recommendations)
to avoid IAM policies with unused and excessive roles in your
environment.

### Protect your endpoints

Consider how an attacker might gain physical access or remote access to your
endpoints, like developer workstations or Compute Engine instances.
While a plan to address the threat of compromised OAuth tokens is
important, also consider how to respond to the threat of how an attacker
can compromise your trusted endpoints in the first place. If an attacker has
access to your trusted endpoints, they can run gcloud CLI
commands or other code directly on the endpoint itself.

Although comprehensive protection for developer workstations is beyond the scope
of this document, evaluate how your security tools and operations can help
protect and monitor your endpoints for compromise. Consider the following
questions:

- How is the physical security of developer workstations protected?
- How do you identify and respond to network breaches?
- How do users get remote access to SSH or RDP sessions?
- How might other persistent credentials like SSH keys or service account keys be compromised?
- Are there workflows that use persistent credentials that could be replaced with short-lived credentials?
- Are there shared devices where someone could read another user's cached gcloud CLI credentials?
- Can a user authenticate with gcloud CLI from an untrusted device?
- How does approved traffic connect to resources inside your VPC Service Control perimeter?

Ensure that your security operations address each of these questions.

### Align your response teams

Ensure in advance that security teams who are responsible for incident response
have appropriate access across the Google Cloud console and the Admin Console.
If separate teams manage the Google Cloud console and the Admin Console, you
might have a delayed response during an incident.

To remove access from a compromised user account, your incident response team
needs an Admin Console role, such as
[User Management Admin](https://support.google.com/a/answer/2405986).
To assess whether suspicious activity has occurred in your Google Cloud
resources, this team might also need
[IAM roles](https://docs.cloud.google.com/iam/docs/understanding-roles),
such as Security Reviewer across all projects or Logs Viewer on a centralized
log sink. The necessary roles for your security team will vary based on the
design and operation of your environment.

## Best practices for remediation after a security incident

After an endpoint is compromised, as part of
[your incident management plan](https://docs.cloud.google.com/architecture/framework/operational-excellence/manage-incidents-and-problems),
determine how to respond to the primary threat of a compromised endpoint and
how to mitigate potential ongoing damage from the secondary threat of
compromised tokens. If an attacker has persistent access to the
developer workstation, they might copy tokens again after the legitimate user
reauthenticates. If you suspect that gcloud CLI tokens might be
compromised, open a ticket with
[Cloud Customer Care](https://docs.cloud.google.com/support/docs/procedures#create_a_support_case)
and complete the recommendations in the following sections. These actions can
help limit the impact of an event like this in your Google Cloud
organization.

The recommendations in this section overlap with the general guidance on
[handling compromised Google Cloud credentials](https://docs.cloud.google.com/docs/security/compromised-credentials),
but focus specifically on the threat of gcloud CLI tokens that are
copied from a compromised endpoint.

### Expire active tokens for all user accounts with Google Cloud session control

If you haven't already enforced
[Google Cloud session control](https://support.google.com/a/answer/9368756),
immediately enable this with a short reauthentication frequency. This control
helps ensure that all refresh tokens expire at the end of the duration that you
define, which limits the duration that an attacker can use the compromised
tokens.

### Manually invalidate tokens for compromised user accounts

Review the guidance for
[handling compromised credentials](https://docs.cloud.google.com/docs/security/compromised-credentials)
for any user identities who could have been compromised. In particular,
[removing gcloud CLI credentials](https://docs.cloud.google.com/docs/security/compromised-credentials#remove_gcloud_credentials)
is the most effective method for a security team to address compromised OAuth
tokens for user identities. To immediately invalidate refresh tokens and access
tokens for the gcloud CLI and force the user to reauthenticate with
their password or security key, remove the gcloud CLI from a user's list
of connected applications.

An individual user can also
[remove gcloud CLI credentials](https://docs.cloud.google.com/docs/security/compromised-credentials#remove_gcloud_credentials_user)
for their individual account.

Other methods, like suspending the user, resetting the user's password, or
resetting sign-in cookies don't specifically address the threat of compromised
OAuth tokens. These methods are generally useful for incident response but don't
invalidate the access tokens the attacker already controls. For example, if you
chose to suspend a user during an investigation but don't revoke
gcloud CLI tokens, the access tokens might still be valid if the
suspended user is restored before the access tokens expire.

### Programmatically invalidate tokens for many user accounts

If you suspect a breach but can't identify which users were affected, consider
revoking active sessions for all users in your
organization more quickly than the reauthentication policy allows.

This approach can be disruptive to legitimate users and terminate long-running
processes that depend on user credentials. If you choose to adopt this approach,
prepare a scripted solution for your security operations center (SOC) to run in
advance and test it with a few users.

The following sample code uses the Workspace Admin SDK to identify all user
identities in your Google Workspace or Cloud Identity account who have
access to the gcloud CLI. If a user has authorized the
gcloud CLI, the script revokes the refresh token and access token
and forces them to reauthenticate with their password or security key. For
instructions on how to enable the Admin SDK API and run this code, see the
[Google Apps Script Quickstart](https://developers.google.com/admin-sdk/directory/v1/quickstart/apps-script).

    /**
     * Remove access to the Google Cloud CLI for all users in an organization
     * @see https://developers.google.com/admin-sdk/directory/reference/rest/v1/tokens
     * @see https://developers.google.com/admin-sdk/directory/reference/rest/v1/users
     * @see https://developers.google.com/apps-script/guides/services/advanced#enabling_advanced_services
     */

    function listUsersAndInvalidate() {
      const users = AdminDirectory.Users.list({
        customer: 'my_customer' // alias to represent your account's customerId
        }).users;
      if (!users || users.length === 0) {
        Logger.log('No users found.');
        return;
      }
      for (const user of users){
        let tokens = AdminDirectory.Tokens.list(user.primaryEmail).items
        if (!tokens || tokens.length === 0) {
          continue;
        }
        for (const token of tokens) {
          if (token.clientId === "32555940559.apps.googleusercontent.com") {
            AdminDirectory.Tokens.remove(user.primaryEmail, token.clientId)
            Logger.log('Invalidated the tokens granted to gcloud for user %s', user.primaryEmail)
          }
        }
      }
    }

### Invalidate and rotate credentials for service accounts

Unlike access tokens that are granted to user identities, access tokens that
are granted to service accounts can't be invalidated through the Admin Console
or commands like
[`gcloud auth revoke`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/revoke).
Additionally, the session duration that you specify in
[Google Cloud session control](https://support.google.com/a/answer/9368756)
applies to user accounts in your Cloud Identity or
Google Workspace directory, but not to service accounts. Therefore, your
incident response for compromised service accounts needs to address both the
persistent key files and the short-lived access tokens.

If you suspect credentials for a service account were compromised,
[disable the service account](https://docs.cloud.google.com/iam/docs/service-accounts-disable-enable#disabling),
[delete service account keys](https://docs.cloud.google.com/iam/docs/keys-create-delete#deleting)
if any exist, and then, after 60 minutes,
[enable the service account](https://docs.cloud.google.com/iam/docs/service-accounts-disable-enable#enabling).
Deleting a service account key can invalidate the long-lived credential so that
an attacker can't request a new access token, but it doesn't invalidate the
access tokens already granted. To ensure access tokens aren't abused within
their 60 minute lifetime, you must disable the service account for 60 minutes.

Alternatively, you can delete and replace the service account to immediately
revoke all short-lived and long-lived credentials, but this might require more
disruptive work to replace the service account in applications.

## What's next

- [Handle compromised Google Cloud credentials](https://docs.cloud.google.com/docs/security/compromised-credentials)
- [Authenticate for the gcloud CLI](https://docs.cloud.google.com/sdk/docs/authenticate)
- [Authenticate as a service account](https://docs.cloud.google.com/docs/authentication/production)
- Explore reference architectures, diagrams, and best practices about Google Cloud. Take a look at our [Cloud Architecture Center](https://docs.cloud.google.com/architecture).