This page describes how to customize Google-owned system components in your
GKE attached clusters by applying custom tolerations and labels. When you
customize these components, you have more precise management of where and how
these components operate within your environment.

## Overview

When you attach a cluster to Google Cloud, by default, Google-owned system
components don't include customizable fields such as tolerations. As a result,
you might experience issues such as Pods failing to schedule on available nodes, which
impact overall cluster attachment and functionality. Additionally, you might require
control over placing and managing components on specific nodes that include the
ability to apply custom labels for organizational and operational purposes.

Customizing system components resolves these issues and lets you have more
control and flexibility over how these components operate in your cluster. You
can apply custom *tolerations* and *labels* to Google-owned system components:

- **Tolerations** provide the ability for Google's components to schedule onto
  nodes that have specific Kubernetes taints, which lets you enforce workload
  separation or use specialized node pools. Custom tolerations directly resolve
  issues where unmodifiable taints prevent proper component placement.

- **Labels** offer a flexible way for you to categorize and identify Google's
  system components according to your own operational standards. Custom labels
  enable better integration with your existing monitoring, logging, and policy
  enforcement tools.

## Restrictions and limitations

The following restrictions and limitations apply:

- You can only add custom tolerations and labels when you [register your cluster](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/attached/aks/how-to/attach-cluster.md.txt#attach_your_aks_cluster).
- You can add up to 10 custom tolerations and 10 custom labels to your cluster.
- You *can't* use the following labels:

  - `name`
  - `component`
  - `app`
  - Any label which includes `k8s.io` or `kubernetes.io` since they belong to reserved labels in Kubernetes
  - Any label which includes `google`
  - Any label which includes `gke.io`

## Add custom tolerations

Google-owned system component Pods always include the following toleration:

      - key: components.gke.io/gke-managed-components
        operator: Exists

To specify custom tolerations, add the `--system-component-tolerations` flag to the
[`gcloud container attached clusters register`](https://docs.cloud.google.com/sdk/gcloud/reference/container/attached/clusters/register) command:

```bash
gcloud container attached clusters register CLUSTER_NAME \
  --location=GOOGLE_CLOUD_REGION \
  ...
  --system-component-tolerations=COMPONENT_TOLERATION \\
```

Replace the following:

- <var translate="no">CLUSTER_NAME</var>: the name of your cluster.
- <var translate="no">GOOGLE_CLOUD_REGION</var>: the Google Cloud region to administer your cluster from.
- <var translate="no">COMPONENT_TOLERATION</var>: the comma-separated list of the
  tolerations that you want to add. You can provide a key, value, operator, and
  effect in the following formats:

  - **For the `Equal` operator** : use the format `key=value:operator:effect`, for example, `workload=hpc:Equal:NoSchedule`. This setting means that the Pod tolerates a taint only if the taint has the exact key and value of `workload=hpc`.
  - **For the `Exists` operator** : use the format `key:operator:effect`, for example, `workload:Exists:NoSchedule`. This setting means that the Pod tolerates any taint with the `workload` key on the node, regardless of the value.
  - **For scheduling on any node** : use the format `:operator:effect`, for example, `:Exists:NoSchedule`. This setting means that the Pod tolerates any taint on the node that has the effect `NoSchedule`, and ignores the key or value of the taint.

  For a list of operators and effects that you
  can use, see [Taints and Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/)
  in the Kubernetes documentation.

## Add custom labels

To specify custom labels, add the `--system-component-labels` flag to the
`gcloud container attached clusters register` command:

```bash
gcloud container attached clusters register CLUSTER_NAME \
  --location=GOOGLE_CLOUD_REGION \
  ...
  --system-component-labels=COMPONENT_LABEL \\
```

Replace the following:

- <var translate="no">CLUSTER_NAME</var>: the name of your cluster.
- <var translate="no">GOOGLE_CLOUD_REGION</var>: the Google Cloud region to administer your cluster from.
- <var translate="no">COMPONENT_LABEL</var>: the comma-separated list of one or more labels that you want to add. You provide a label and value in the format of `key=value`. For example, `env=production,region=us-east-1`. Although every label must have a key, the value associated with that key can be empty. For example, `backend=""`.

## What's next

- Learn more about [Taints and Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) in the Kubernetes documentation.
- Learn how to [attach your AKS cluster](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/attached/aks/how-to/attach-cluster.md.txt).