Planning your Google Kubernetes Engine (GKE) cluster configuration before you
install Config Sync helps ensure reliable policy and configuration
synchronization across your environments. Your clusters must meet specific
requirements for the cluster mode of operation, fleet registration,
Identity and Access Management (IAM) permissions, platform versions, authentication,
networking, and node architecture.

For general best practices when planning your GKE clusters, see
[About cluster configuration choices](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/configuration-overview.md.txt).

## Choose a GKE mode of operation

Config Sync supports both Autopilot and Standard
GKE clusters.

To help make an informed choice about which mode of operation to use, read
[GKE modes of operation](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/choose-cluster-mode.md.txt).

## Register your clusters to a fleet

Config Sync requires that your clusters are registered to a fleet.

If you plan to install and configure Config Sync with the Google Cloud console,
you can register your cluster at the same time you install Config Sync.

Otherwise, you must [register your cluster to a fleet](https://docs.cloud.google.com/kubernetes-engine/fleet-management/docs/register/gke.md.txt)
before enabling Config Sync.

## Grant required permissions

Because Config Sync requires that clusters are registered to a fleet,
you might require additional permissions when installing Config Sync.


To get the permissions that
you need to register clusters to a fleet,

ask your administrator to grant you the
[Fleet Admin (formerly GKE Hub Admin)](https://docs.cloud.google.com/iam/docs/roles-permissions/gkehub.md.txt#gkehub.admin) (`roles/gkehub.admin`) IAM role on your project.


For more information about granting roles, see [Manage access to projects, folders, and organizations](https://docs.cloud.google.com/iam/docs/granting-changing-revoking-access.md.txt).


You might also be able to get
the required permissions through [custom
roles](https://docs.cloud.google.com/iam/docs/creating-custom-roles) or other [predefined
roles](https://docs.cloud.google.com/iam/docs/roles-overview#predefined).

## Resource requirements with Autopilot mode

GKE Autopilot mode automatically modifies resource
requests to maintain workload stability. To understand how to plan for those
requests, review the GKE documentation for
[Autopilot resource requests](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/autopilot-resource-requests.md.txt).

> [!TIP]
> **Tip:** You can check if Autopilot adjusted your resource requirements by looking for the `autopilot.gke.io/resource-adjustment` annotation on your workloads.

Due to the way Autopilot modifies resource requests, Config Sync
makes the following adjustments:

- Adjusts [user-specified resource override limits to match requests](https://docs.cloud.google.com/kubernetes-engine/config-sync/docs/reference/rootsync-reposync-fields.md.txt#override-resources).
- Applies overrides only when there are one or more resource requests higher than the corresponding adjusted output declared in the annotation, or there are resource requests lower than the corresponding input declared in the annotation.

## Supported GKE platforms and versions

To use Config Sync, your cluster must be on a GKE
[supported version](https://docs.cloud.google.com/kubernetes-engine/versioning.md.txt#available_versions).

## Workload Identity Federation for GKE

Workload Identity Federation for GKE is the recommended way to securely connect to Google Cloud
services. Workload Identity Federation for GKE is enabled by default on Autopilot
clusters.

If you want to use [fleet packages](https://docs.cloud.google.com/kubernetes-engine/config-sync/docs/how-to/fleet-packages.md.txt) with Config Sync,
Workload Identity Federation for GKE is required.

If you installed Config Sync on
[GKE attached clusters](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/attached.md.txt),
you can't use Active Directory with
Workload Identity Federation. This limitation exists because Config Sync uses
connect gateway to connect to GKE attached clusters and
connect gateway doesn't support this feature.

## Networking

The following section lists some of the changes you might need to make to
your GKE cluster, depending on your networking settings.

For more information about GKE networking choices, see
[Network overview](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/network-overview).

### Private clusters

If you use GKE clusters with [private nodes](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/network-isolation.md.txt#enable-private-nodes), you should configure your clusters in one of
the following ways to ensure Config Sync has access and can authenticate to
your :

- Configure [Cloud NAT](https://docs.cloud.google.com/nat/docs/overview.md.txt) to permit egress from private
  GKE nodes. For details, see [Example GKE
  setup](https://docs.cloud.google.com/nat/docs/gke-example).

- Enable [Private Google Access](https://docs.cloud.google.com/vpc/docs/configure-private-google-access.md.txt) to connect to the set of external IP
  addresses used by [Google APIs and services](https://developers.google.com/apis-explorer/).

  > [!NOTE]
  > **Note:** [Private Google Access doesn't support SSH on port 2022](https://docs.cloud.google.com/vpc/docs/configure-private-google-access.md.txt#config). However, when you grant Config Sync read-only access to Git, you can use `cookiefile` credentials.

### Public clusters

If you use public clusters, but have strict VPC Firewall requirements that block
any unnecessary traffic, you must [Create firewall rules](https://docs.cloud.google.com/vpc/docs/using-firewalls#creating_firewall_rules) to permit the
following traffic:

- **TCP**: Allow ingress and egress on port 53 and 443
- **UDP**: Allow egress on port 53

If you don't include these rules, Config Sync doesn't sync correctly and
`nomos status` reports the following error:

`Error: KNV2004: unable to sync repo Error in the git-sync container`

## Cloud Source Repositories with Compute Engine default service account authentication

If you are using Config Sync to connect to Cloud Source Repositories and
Workload Identity Federation for GKE is not enabled, you can use the Compute Engine default
service account to authenticate. You must use
[access scopes](https://docs.cloud.google.com/compute/docs/access/service-accounts.md.txt#accesscopesiam) with read-only scopes for the nodes in the cluster.

> [!NOTE]
> **Note:** If you don't use custom IAM service accounts to create your Config Sync clusters or node pools, ensure that the default Compute Engine service account in your project has the required permissions for GKE. In organizations that enforce the [`iam.automaticIamGrantsForDefaultServiceAccounts` organization policy constraint](https://docs.cloud.google.com/resource-manager/docs/organization-policy/restricting-service-accounts#disable_service_account_default_grants), the default Compute Engine service account won't automatically get the required permissions for GKE. This constraint is enforced by default for organizations that were created on or after May 3, 2024. For details, see [Default GKE node service account](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/service-accounts.md.txt#default-gke-service-agent).

You can add the read-only scope for Cloud Source Repositories by including
`cloud-source-repos-ro` in the `--scopes` list specified at cluster creation
time, or by using the `cloud-platform` scope at cluster creation time. For
example:

```
gcloud container clusters create CLUSTER_NAME --scopes=cloud-platform
```

Replace `CLUSTER_NAME` with the name of your cluster.

You cannot modify access scopes after you create a node pool.
However, you can create a new node pool with the proper access scope while
using the same cluster. The default `gke-default` scope does *not* include
`cloud-source-repos-ro`.

## Arm nodes

Config Sync can run only on x86-based nodes, not Arm nodes. However, if you
need to run Config Sync on a cluster with multiple architectures, take the
following action, depending on your cluster type:

- **GKE on AWS** or **GKE on Azure** : add a [taint](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to your Arm nodes, to avoid scheduling Pods onto your Arm nodes without a corresponding toleration.
- **GKE**: GKE adds a default taint to ensure workloads without the corresponding toleration aren't scheduled there. No additional action needed.

## What's next

[Install Config Sync](https://docs.cloud.google.com/kubernetes-engine/config-sync/docs/how-to/installing-config-sync.md.txt).