This page describes how to rotate keys for the following service accounts:

- [Component access](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/service-accounts.md.txt#component_access_service_account)
- [Connect-register](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/service-accounts.md.txt#connect_register_service_account)
- [Logging-monitoring](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/service-accounts.md.txt#logging_monitoring_service_account)
- [Audit logging](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/service-accounts.md.txt#audit_logging_service_account)

> [!IMPORTANT]
> **Important:** The capabilities described in this document aren't fully supported for 1.31 or 1.32 clusters that have advanced clusters enabled. For details about what is or isn't supported for updating credentials for advanced clusters, see [Advanced cluster differences for rotating service account
> keys](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/rotating-service-account-keys#adv-cluster-diffs).

To rotate your service account keys:

1. Create a directory to store a backup of your current secrets:

   ```
   mkdir backup
   ```
2. Note the following information for the relevant service account:

   **Component access**

   | Cluster | Secret | Namespace |
   |---|---|---|
   | Admin | admin-cluster-creds | kube-system |
   | Admin | user-cluster-creds | <var translate="no">CLUSTER_NAME</var>-gke-onprem-mgmt |
   | Admin | private-registry-creds | kube-system |
   | User | private-registry-creds | kube-system |

   - If you aren't using a [private registry](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/admin-cluster-configuration-file-latest.md.txt#privateregistry-section), the `private-registry-creds` Secret holds the key for your component access service account.
   - If you are using a private registry, the `private-registry-creds` Secret holds the credentials for your private registry, **not** the component access service account key.

   **Connect-register**

   | Cluster | Secret | Namespace |
   |---|---|---|
   | Admin | admin-cluster-creds | kube-system |
   | Admin | user-cluster-creds | <var translate="no">CLUSTER_NAME</var>-gke-onprem-mgmt |

   **Logging-monitoring**

   | Cluster | Secret | Namespace |
   |---|---|---|
   | Admin | admin-cluster-creds | kube-system |
   | Admin | user-cluster-creds | <var translate="no">CLUSTER_NAME</var>-gke-onprem-mgmt |
   | User | google-cloud-credentials | kube-system |
   | User | stackdriver-service-account-key | knative-serving |

   **Audit logging**

   | Cluster | Secret | Namespace |
   |---|---|---|
   | Admin | admin-cluster-creds | kube-system |
   | Admin | user-cluster-creds | <var translate="no">CLUSTER_NAME</var>-gke-onprem-mgmt |
   | Admin | kube-apiserver | <var translate="no">CLUSTER_NAME</var> |

   **Stackdriver**

   | Cluster | Secret | Namespace |
   |---|---|---|
   | Admin | admin-cluster-creds | kube-system |
   | Admin | user-cluster-creds | <var translate="no">CLUSTER_NAME</var>-gke-onprem-mgmt |
   | User | google-cloud-credentials | kube-system |
   | User | stackdriver-service-account-key | knative-serving |

3. Create a backup of each secret using the following command:

   ```
   kubectl get secret SECRET --namespace NAMESPACE \
       --kubeconfig KUBECONFIG -o json > backup/SECRET-NAMESPACE.json
   ```

   Replace the following:
   - `NAMESPACE`: the namespace where the secret is located. For example, `kube-system`.
   - `KUBECONFIG`: the path to the kubeconfig file for the admin or user cluster.
   - `SECRET`: the name of the secret. For example, `admin-cluster-creds`.

   For example, run the following commands for the audit logging service
   account:

   ```
   kubectl get secret admin-cluster-creds --namespace kube-system \
           --kubeconfig KUBECONFIG -o json > backup/admin-cluster-creds-kube-system.json

   kubectl get secret user-cluster-creds --namespace NAMESPACE \
           --kubeconfig KUBECONFIG -o json > backup/user-cluster-creds-NAMESPACE.json

   kubectl get secret kube-apiserver --namespace NAMESPACE \
           --kubeconfig KUBECONFIG -o json > backup/kube-apiserver-NAMESPACE.json
   ```
4. To create a new service account key file, run the following command:

   ```
   gcloud iam service-accounts keys create NEW_KEY_FILE --iam-account IAM_ACCOUNT
   ```

   Replace the following:
   - `NEW_KEY_FILE`: the name for your new service account key file
   - `IAM_ACCOUNT`: the email address of the service account
5. In the admin cluster configuration file, find the
   `componentAccessServiceAccountKeyPath` field, the `gkeConnect` section, the
   `stackdriver` section, and the `cloudAuditLogging` section. In those places,
   replace the paths to the service account key files.

6. In the user cluster configuration file, find the
   `componentAccessServiceAccountKeyPath` field, the `gkeConnect` section, the
   `stackdriver` section, and the `cloudAudigLogging` section.
   In those places, replace the paths to the service
   account key files.

7. Save the changes you made by running the following commands:

   You can rotate the keys for one component at a time, or you can rotate all
   keys at once by setting the component to `sakeys`.

   ```
   gkectl update credentials COMPONENT \
       --kubeconfig ADMIN_CLUSTER_KUBECONFIG \
       --config ADMIN_CLUSTER_CONFIG \
       --admin-cluster

   gkectl update credentials COMPONENT \
       --kubeconfig ADMIN_CLUSTER_KUBECONFIG \
       --config USER_CLUSTER_CONFIG
   ```

   Replace the following;
   - `COMPONENT`: one of:

     - `componentaccess`
     - `register`
     - `cloudauditlogging`
     - `stackdriver`
     - `sakeys` (rotates all component keys)
   - `ADMIN_CLUSTER_KUBECONFIG`: the path to the
     kubeconfig file for the admin cluster.

   - `ADMIN_CLUSTER_CONFIG`: the path to the admin
     cluster configuration file.

   - `USER_CLUSTER_CONFIG`: the path to the user
     cluster configuration file.

## Node re-creation

Some service account key rotations may take longer time because node
re-creation is required:

<br />

| **Service account** | **Nodes re-creation required** |
|---|---|
| Component access | If using Artifact Registry: Yes If using a private registry: No |
| Audit logging | Admin cluster: Yes but only control-plane nodes User cluster using kubeception: No User cluster using Controlplane V2: Yes but only control plane nodes |
| Logging-monitoring | No |
| Connect-register | No |

<br />

For a key rotation that requires nodes to be re-created, the nodes are replaced
in a rolling update process; that is, the nodes are re-created one by one.

The possible downtime during a key rotation is similar to the downtime for a
cluster upgrade. For details, see
[Downtime during upgrades](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/upgrading.md.txt#downtime_upgrades).

## Restoring backups

If you need to restore the backups of the secrets you made earlier, run the
following command:

```
kubectl apply -f backup/
```

## Advanced cluster differences for rotating service account keys

The `gkectl update credentials` command is only partially supported for advanced
clusters. The following table lists command variations that are and aren't
supported:

| Command | Description | Support |
|---|---|---|
| `gkectl update credentials certificate-authorities rotate` | Rotate certificate authorities. | Supported |
| `gkectl update credentials certificate-authorities status` | ([Preview](https://cloud.google.com/products/#product-launch-stages)) View certificate authorities rotation status. | Supported |
| `gkectl update credentials certificate-authorities update-kubeconfig` | Download a kubeconfig file with latest client certificate. | Supported |
| `gkectl update credentials cloudauditlogging` | Update Cloud Audit Logs credentials. | Not Supported |
| `gkectl update credentials componentaccess` | Update the component access service account key credentials for clusters created with Google Distributed Cloud for VMware. | Supported |
| `gkectl update credentials f5bigip` | Update F5 BIG-IP load balancer credentials. | Not Needed |
| `gkectl update credentials ksa-signing-key rotate` | Rotate the Kubernetes service account (KSA) token signing key. | Not Supported |
| `gkectl update credentials ksa-signing-key status` | View the status of an ongoing KSA token signing key rotation. | Not Supported |
| `gkectl update credentials` | Update cluster credentials. | Supported |
| `gkectl update credentials privateregistry` | Update private registry credentials for a cluster. | Supported |
| `gkectl update credentials register` | Update register service account key credentials. | Supported |
| `gkectl update credentials sakeys` | Update service account keys credentials for a cluster. | Supported |
| `gkectl update credentials stackdriver` | ([Preview](https://cloud.google.com/products/#product-launch-stages)) Update Google Cloud Observability credentials. | Supported |
| `gkectl update credentials vsphere` | Update vSphere credentials for a cluster. | Supported |