Learn how to configure GKE on AWS to use
[OpenID Connect](https://developers.google.com/identity/protocols/OpenIDConnect)
(OIDC) for authentication to user clusters. This topic covers the process to
configure GKE on AWS with any OpenID provider.

> [!NOTE]
> **Note:** For new clusters running Kubernetes 1.21 or higher, we recommend using [GKE Identity Service](https://docs.cloud.google.com/anthos/identity) for OIDC authentication instead of the configuration described in this page. To set up GKE Identity Service, [Create a user cluster](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/creating-user-cluster.md.txt), and then follow the steps in [Configuring clusters for GKE Identity Service with OIDC](https://docs.cloud.google.com/anthos/identity/setup/per-cluster) or [Setting up GKE Identity Service for a fleet](https://docs.cloud.google.com/anthos/identity/setup/fleet).

To upgrade a cluster that uses OIDC authentication to Kubernetes 1.21, see
[Upgrade to 1.21](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/oidc.md.txt#upgrade-121).

For an overview of the GKE on AWS authentication flow, see
[Authentication](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/concepts/authentication.md.txt).

## Overview

GKE on AWS supports OIDC as one of the authentication mechanisms for
interacting with a user cluster's Kubernetes API server. With OIDC, you can
manage access to Kubernetes clusters by using the standard procedures in your
organization for creating, enabling, and disabling user accounts.

## Before you begin

- This topic assumes you are familiar with the following authentication and
  OpenID concepts:

  - [OAuth 2.0](https://oauth.net/2/)
  - [OpenID Connect](https://openid.net/connect/)
  - [scopes](https://openid.net/specs/openid-connect-core-1_0.html#ScopeClaims)
  - [claims](https://openid.net/specs/openid-connect-core-1_0.html#IDToken)
- The Google Cloud CLI must be installed on each developer's local machine.

- Headless systems are unsupported. To authenticate, you must
  open a browser on the local machine running the gcloud CLI. The
  browser then prompts you to authorize your user account.

- To authenticate through the Google Cloud console, each cluster that you want to
  configure for OIDC authentication must be
  [registered with Google Cloud](https://docs.cloud.google.com/anthos/multicluster-management/connect/registering-a-cluster).

## Personas

This topic refers to three personas:

- *Organization administrator*: This person chooses an OpenID provider and
  registers client applications with the provider.

- *Cluster administrator*: This person creates one or more user clusters and
  creates authentication configuration files for developers who use the
  clusters.

- *Developer*: This person runs workloads on one or more clusters and uses OIDC
  to authenticate.

## Choose an OpenID provider

This section is for [organization administrators](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/oidc.md.txt#personas).

You can use any OpenID provider of your choice. For a list of certified
providers, see
[OpenID Certification](https://openid.net/certification/).

## Create redirect URLs

This section is for [organization administrators](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/oidc.md.txt#personas).

The OpenID provider uses redirect URLs to return ID tokens. You must create
redirect URLs for both the gcloud CLI and the
Google Cloud console.

### Set the gcloud CLI redirect URL

When you configure your OpenID provider, specify your CLI redirect
URL as
`http://localhost:PORT/callback`

Replace <var translate="no">PORT</var> with your port number greater than 1024.

### Set the Google Cloud console redirect URL

The redirect URL for the Google Cloud console is:

`https://console.cloud.google.com/kubernetes/oidc`

When you configure your OIDC provider, specify
`https://console.cloud.google.com/kubernetes/oidc` as one of your redirect URLs.

## Register your client applications with the OpenID provider

This section is for [organization administrators](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/oidc.md.txt#personas).

Before your developers can use the Google Cloud CLI or the Google Cloud console
with your OpenID provider, you need to register those two clients with the
OpenID provider. Registration includes these steps:

- Learn your provider's issuer URI. The gcloud CLI or
  Google Cloud console sends authentication requests to this URI.

- Configure your provider with the redirect URL, including your port number, for
  the gcloud CLI.

- Configure your provider with the redirect URL for the Google Cloud console,
  `https://console.cloud.google.com/kubernetes/oidc`.

- Create a single
  [client ID](https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/)
  that your provider uses to identify both the Google Cloud CLI and the
  Google Cloud console.

- Create a single
  [client secret](https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/)
  that the gcloud CLI and the Google Cloud console use to
  authenticate to the OpenID provider.

- Create a custom scope that the gcloud CLI or Google Cloud console
  can use to request the user's security groups.

- Create a custom claim name that the provider uses to return the user's
  security groups.

## Configure your cluster

This section is for [cluster administrators](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/oidc.md.txt#personas).

To configure OIDC authentication, you need to configure your user cluster's
[AWSCluster resource](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/reference/awscluster.md.txt) with authentication
details for a cluster. Details from the AWSCluster are used to configure OIDC
for both the Google Cloud console and the Authentication Plugin for
GKE. The configuration includes the following OIDC information:

```yaml
authentication:
  awsIAM:
    adminIdentityARNs:
      - AWS_IAM_ARN
  oidc:
  -   certificateAuthorityData: CERTIFICATE_STRING
      clientID: CLIENT_ID
      clientSecret: CLIENT_SECRET 
      extraParams:  EXTRA_PARAMS
      groupsClaim:  GROUPS_CLAIM
      groupPrefix:  GROUP_PREFIX
      issuerURI:  ISSUER_URI
      kubectlRedirectURI:  KUBECTL_REDIRECT_URI
      scopes:  SCOPES
      userClaim:  USER_CLAIM
      userPrefix:  USER_PREFIX
```

### Authentication fields

The following table describes the fields of the
`authentication.awsIAM.adminIdentityARNs` object.

| Field | Required | Description | Format |
|---|---|---|---|
| adminIdentityARNs | Yes, if configuring OIDC. | Amazon Resource Name (ARN) of the AWS IAM identities (users or roles) granted cluster administrator access by GKE on AWS. Example: `arn:aws:iam::123456789012:group/Developers` | String |

| Field | Required | Description | Format |
|---|---|---|---|
| certificateAuthorityData | No | A base64-encoded [PEM-encoded certificate](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail) for the OIDC provider. To create the string, encode the certificate, including headers, into base64. Include the resulting string in `certificateAuthorityData` as a single line. Example: `certificateAuthorityData: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tC...k1JSUN2RENDQWFT==` | String |
| clientID | Yes | ID for the client application that makes authentication requests to the OpenID provider. | String |
| clientSecret | No | Shared secret between OIDC client application and OIDC provider. | String |
| extraParams | No | Additional key-value parameters to send to the OpenID provider. If you are authorizing a group, pass in `resource=token-groups-claim`. If your authorization server prompts for consent, for authentication with Microsoft Azure and Okta, set `extraParams` to `prompt=consent`. For Cloud Identity, set `extraParams` to `prompt=consent,access_type=offline`. | Comma-delimited list |
| groupsClaim | No | [JWT](https://tools.ietf.org/html/rfc7519) claim that the provider uses to return your security groups. | String |
| groupPrefix | No | Prefix prepended to group claims to prevent clashes with existing names. For example, If you have two groups named `foobar` add a prefix `gid-`, the result group is `gid-foobar`. | String |
| issuerURI | Yes | URL where authorization requests are sent to your OpenID, such as `https://example.com/adfs`. The Kubernetes API server uses this URL to discover public keys for verifying tokens. The URI must use HTTPS. | URL String |
| kubectlRedirectURI | Yes | The redirect url \`kubectl\` uses for authorization. | URL String |
| scopes | Yes | Additional scopes to send to the OpenID provider. Microsoft Azure and Okta require the `offline_access` scope. | Comma-delimited list |
| userClaim | No | JWT claim to use as the username. You can choose other claims, such as email or name, depending on the OpenID provider. However, claims other than email are prefixed with the issuer URL to prevent naming clashes. | String |
| userPrefix | No | Prefix prepended to username claims to prevent clashes with existing names. | String |

### Example: Authorizing users and groups

Many providers encode user-identifying properties, such as email and user
IDs, in a token. However, these properties have implicit risks for
authentication policies:

- User IDs can make policies difficult to read and audit.
- Using email addresses can create both an availability risk (if a user changes their primary email) and a security risk (if an email can be re-assigned).

Instead of assigning user IDs, we recommend group policies, which can be
both persistent and easier to audit.

Suppose your provider creates identity tokens that include the following
fields:

```
{
  'iss': 'https://server.example.com'
  'sub': 'u98523-4509823'
  'groupList': ['developers@example.corp', 'us-east1-cluster-admins@example.corp']
  ...
}
```

Given this token format, you'd populate your configuration file's
`oidc` specification like so:

```yaml
issuerURL: 'https://server.example.com'
username: 'sub'
usernamePrefix: 'uid-'
group: 'groupList'
groupPrefix: 'gid-'
extraParams: 'resource=token-groups-claim'
...
```

After you've created your user cluster, you use Kubernetes role-based access
control (RBAC) to grant privileged access to the authenticated users.

In the following example, you create a `ClusterRole` that grants its users read-
only access to the cluster's Secrets, and create a `ClusterRoleBinding` resource
to bind the role to the authenticated group.

1. Define a `ClusterRole`. Copy the following YAML into a file named
   `secret-reader-role.yaml`.

   ```yaml
   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: secret-reader
   rules:
   - apiGroups: [""]
     # The resource type for which access is granted
     resources: ["secrets"]
     # The permissions granted by the ClusterRole
     verbs: ["get", "watch", "list"]
   ```
2. Define a `ClusterRoleBinding`. Copy the following YAML into a file named
   `secret-reader-admins.yaml`.

   ```yaml
   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: read-secrets-admins
   subjects:
     # Allows anyone in the "us-east1-cluster-admins" group to
     # read Secrets in any namespace within this cluster.
   - kind: Group
     name: gid-us-east1-cluster-admins # Name is case sensitive
     apiGroup: rbac.authorization.k8s.io
     # Allows this specific user to read Secrets in any
     # namespace within this cluster
   - kind: User
     name: uid-u98523-4509823
     apiGroup: rbac.authorization.k8s.io
   roleRef:
     kind: ClusterRole
     name: secret-reader
     apiGroup: rbac.authorization.k8s.io
   ```
3. Apply `secret-reader-role.yaml` and `secret-reader-admins.yaml` to your
   cluster with `kubectl`.

   ```bash
   env HTTPS_PROXY=http://localhost:8118 \
     kubectl apply -f secret-reader-role.yaml && \
     kubectl apply -f secret-reader-admins.yaml
   ```

   Users granted access in `read-secrets-admins` now have access to read
   Secrets in your cluster.

## Create a login config

This section is for [cluster administrators](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/oidc.md.txt#personas).

After you create a user cluster, you need to generate a configuration file
for the cluster using `gcloud anthos create-login-config`.

1. From your `anthos-aws` directory, use
   `anthos-gke` to switch context to your user cluster.

   ```sh
   cd anthos-aws
   env HTTPS_PROXY=http://localhost:8118 \
     anthos-gke aws clusters get-credentials CLUSTER_NAME
   ```
   Replace <var translate="no">CLUSTER_NAME</var> with your user cluster name.

   <br />

2. Create the configuration with `gcloud anthos`.

   ```bash
   gcloud anthos create-login-config --kubeconfig usercluster-kubeconfig
   ```

   Replace <var translate="no">usercluster-kubeconfig</var> with the path to your user cluster's
   `kubeconfig` file. On Linux and macOS, by default this file is at
   `~/.kube/config`.

This command generates a file (`kubectl-anthos-config.yaml`) containing the
configuration information your developers use to authenticate to the
cluster with the gcloud CLI. You should not modify this file.

To understand more about the contents of `kubectl-anthos-config.yaml`, see
the [appendix](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/oidc.md.txt#appendix-config).

### Distribute the login config

Distribute the config file to users that need to authenticate to your user
clusters. You can distribute the config by:

- Placing the file in the default directory.
- Securely distributing the file.
- Hosting the file on an HTTPS server.

#### Login config default directories

The default locations for storing the configuration file for each OS are as
follows:

Linux
:   `$HOME/.config/google/anthos/kubectl-anthos-config.yaml`, where `$HOME` is
    the user's home directory.

macOS
:   `$HOME/Library/Preferences/google/anthos/kubectl-anthos-config.yaml`, where
    `$HOME` is the user's home directory.

Windows
:   `%APPDATA%/google/anthos/kubectl-anthos-config.yaml`, where `%APPDATA%` is
    the user's application data directory.

After the login config has been distributed, your developers are ready to
configure the gcloud CLI to access the cluster.

## Modify your cluster after upgrading to Kubernetes 1.21

After you upgrade your cluster to Kubernetes 1.21, you need to configure
GKE Identity Service and remove your OIDC information from your cluster's
configuration. To update the configuration, perform the following steps:

1. Follow the steps in
   [Upgrade your cluster](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/upgrading.md.txt).

2. From your `anthos-aws` directory, use
   `anthos-gke` to switch context to your user cluster.

   ```sh
   cd anthos-aws
   env HTTPS_PROXY=http://localhost:8118 \
     anthos-gke aws clusters get-credentials CLUSTER_NAME
   ```
   Replace <var translate="no">CLUSTER_NAME</var> with your user cluster name.

   <br />

3. Open the manifest that contains the AWSCluster in a text editor.
   Keep the file open and use the values of the `oidc` object to follow the
   steps in
   [Configuring clusters for GKE Identity Service](https://docs.cloud.google.com/anthos/identity/setup/per-cluster#configure_clusters).

4. From your `anthos-aws` directory, use
   `anthos-gke` to switch context to your management service.

   ```sh
   cd anthos-aws
   anthos-gke aws management get-credentials
   ```

   > [!NOTE]
   > **Note:** `anthos-gke` uses the credentials you specified when you ran `aws configure`.

   <br />

5. Open the YAML file that created your AWSCluster in a text editor.
   If you do not have your initial YAML file, you can use `kubectl edit`.

   ### Edit YAML

   If you followed the instructions in
   [Creating a user cluster](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/creating-user-cluster.md.txt), your YAML file is named
   `cluster-0.yaml`. Open this file in a text editor.

   ### kubectl edit

   To use `kubectl edit` to edit your AWSCluster, run the following
   command:

   ```bash
   env HTTPS_PROXY=http://localhost:8118 \
     kubectl edit awscluster cluster-name
   ```

   Replace <var translate="no">cluster-name</var> with your AWSCluster. For
   example, to edit the default cluster, `cluster-0`, run the following
   command:

   ```bash
   env HTTPS_PROXY=http://localhost:8118 \
     kubectl edit awscluster cluster-0
   ```
6. Delete the `oidc` object from your cluster's manifest.

7. Save the file. If you are using `kubectl edit`, `kubectl` applies the
   changes automatically. If you are editing the YAML file, apply it to your
   management service with the following command:

   ```bash
   env HTTPS_PROXY=http://localhost:8118 \
     kubectl apply -f cluster-0.yaml
   ```

   The management service then updates your AWSCluster.

## Configuring gcloud to access your cluster

This section is for [developers or cluster administrators](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/oidc.md.txt#personas).

### Prerequisites

To complete this section, you must complete the following:

- A [login config](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/oidc.md.txt#distribute-login-config).
- An updated version of the gcloud CLI with the `anthos-auth`
  components.

  ```bash
  gcloud components update
  gcloud components install anthos-auth
  ```
- Verify that the gcloud CLI was installed successfully by running the
  following command, which should respond with details about the required
  arguments and available options.

  ```bash
  gcloud anthos auth
  ```

### Authenticate to you cluster

You can authenticate to your cluster the following ways:

- With the gcloud CLI on your local machine.
- With the gcloud CLI on a remote machine using an SSH tunnel.
- With Connect on the Google Cloud console.

### gcloud local

Use `gcloud anthos auth login` to authenticate to your cluster with your login
config.

If you placed the login config in the default location and have the cluster
name configured, you can use `gcloud anthos auth login` with no options. You can
also configure the cluster, user, and other authentication details with optional
parameters.

### Default

```bash
gcloud anthos auth login --cluster CLUSTER_NAME
```

Replace <var translate="no">CLUSTER_NAME</var> with a fully qualified cluster name. For
example,
`projects/my-gcp-project/locations/global/memberships/cluster-0-0123456a`.

### Optional parameters

`gcloud anthos auth login` supports the following optional parameters:

```bash
gcloud anthos auth login --cluster CLUSTER_NAME \
--user USERNAME --login-config ANTHOS_CONFIG_YAML \
--login-config-cert LOGIN_CONFIG_CERT_PEM \
--kubeconfig=KUBECONFIG --dry-run
```

The parameters are described in the following table.

| Parameter | Description |
|---|---|
| <var translate="no">cluster</var> | The name of the cluster to authenticate to. Defaults to the cluster in \`kubectl-anthos-config.yaml\`. |
| <var translate="no">user</var> | Username for credentials in `kubeconfig`. Defaults to `{cluster-name}-anthos-default-user`. |
| <var translate="no">login-config</var> | Either the path to the configuration file generated by the cluster admin for the developer or a URL hosting the file. Defaults to [`kubectl-anthos-config.yaml`](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/oidc.md.txt#distribute-login-config). |
| <var translate="no">login-config-cert</var> | If using a URL for <var translate="no">login-config</var>, the path to the CA certificate file for making HTTPS connections. |
| <var translate="no">kubeconfig</var> | Path to the kubeconfig file that contains tokens. Defaults to `` $HOME/.kube/config` ``. |
| dry-run | Test your command-line options without changing your configuration or cluster. |

The `gcloud anthos login` command launches a browser that asks the user to
log in with their enterprise credentials, performs the OIDC credential
exchange, and acquires the relevant tokens. The gcloud CLI then
writes the tokens to a `kubeconfig` file. `kubectl` uses this file to
authenticate to the user cluster.

To verify that the authentication was successful, run any `kubectl` command with
your `kubeconfig` file:

```bash
env HTTPS_PROXY=http://localhost:8118 \
  kubectl get nodes --kubeconfig my.kubeconfig
```

### gcloud tunnel

If you want to authenticate to a user cluster
from a remote machine, you can perform the authentication using an SSH
tunnel. To use a tunnel, your authentication configuration file must
be on the remote machine, and you must be able to reach your Open ID provider
from your local machine.

On your local machine, run the following command:

```
ssh USERNAME@REMOTE_MACHINE -L LOCAL_PORT:localhost:REMOTE_PORT
```

Replace the following:

- <var translate="no">USERNAME</var> with a user that has SSH access to the remote
  machine.

- <var translate="no">REMOTE_MACHINE</var> with the remote machine's hostname or IP
  address.

- <var translate="no">LOCAL_PORT</var> is an available port on your local machine
  that `ssh` uses to tunnel to the remote machine.

- <var translate="no">REMOTE_PORT</var> is the port that you configured for your OIDC
  redirect URL. The port number is part of the `kubectlRedirectURI` field of
  your authentication configuration file.

In your SSH shell, run the following command to initiate authentication:

```
gcloud anthos auth login --login-config AUTH_CONFIG_FILE
```

Replace <var translate="no">AUTH_CONFIG_FILE</var> with the path of your authentication
configuration file on the remote machine. The gcloud CLI runs a
web server on the remote machine.

On your local machine, in a browser, go to
http://localhost:<var translate="no">LOCAL_PORT</var>/login and follow the
OIDC login flow.

The `kubeconfig` file on your remote machine now has the token to access the
user cluster.

In your SSH shell, verify that you have access to the user cluster:

```sh
kubectl --kubeconfig USER_CLUSTER_KUBECONFIG get nodes
```

### Console

You can authenticate with the Google Cloud console, initiate the authentication
flow from the Kubernetes clusters page in the Google Cloud console:

1. Open the Google Cloud console:

   [Visit the Kubernetes clusters page](https://console.cloud.google.com/kubernetes/list)
2. Locate your GKE on AWS cluster in the list and then click
   **Login**.

3. Select **Authenticate with the Identity Provider configured for the
   cluster** , and then click **LOGIN**.

   You are redirected to your identity provider, where you might need to
   log in or consent to the Google Cloud console accessing your account.
   You are then redirected back to the **Kubernetes clusters**
   page on the Google Cloud console.

## Updating OIDC configuration

To update the OIDC configuration on your cluster, use the `kubectl edit`
command.

```bash
env HTTPS_PROXY=http://localhost:8118 \
  kubectl edit clientconfigs -n kube-public default
```

The `kubectl` tool loads the ClientConfig resource in your default editor. To
update the configuration, save the file. The `kubectl` tool updates the
ClientConfig resource on your cluster.

For information on the contents of the ClientConfig resource, see the following
section.

## Appendix: Example login config

An example `kubectl-anthos-config.yaml` follows. This example is included for
understanding its contents. You should always generate the file with
`gcloud anthos create-login-config`.

```yaml
apiVersion: authentication.gke.io/v2alpha1
kind: ClientConfig
metadata:
 name: default
 namespace: kube-public
spec:
  authentication:
  - name: oidc
    oidc:
      clientID: CLIENT_CONFIG
      clientSecret: CLIENT_SECRET
      extraParams: resource=k8s-group-claim,domain_hint=consumers
      certificateAuthorityData:   CERTIFICATE_STRING
      issuerURI: https://adfs.contoso.com/adfs
      kubectlRedirectURI: http://redirect.kubectl.com/
      scopes: allatclaim,group
      userClaim: "sub"
      groupsClaim: "groups"
    proxy: PROXY_URL #Optional
  certificateAuthorityData: CERTIFICATE_AUTHORITY_DATA
  name: projects/my-project/locations/global/membership/cluster-0
  server: https://192.168.0.1:PORT
  preferredAuthentication: oidc
```

For explanations of the field contents, see
[Authentication fields](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/how-to/oidc.md.txt#authentication_fields).

## What's next

[Deploy your first workload](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/aws/previous-generation/quickstart.md.txt) to GKE on AWS.