> [!WARNING]
>
> **Preview**
>
>
> This feature is
>
> subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the
> [Service Specific
> Terms](https://docs.cloud.google.com/terms/service-terms#1).
>
> Pre-GA features are available "as is" and might have limited support.
>
> For more information, see the
> [launch stage descriptions](https://cloud.google.com/products/#product-launch-stages).

Topology domains in Google Distributed Cloud software only for VMware let you spread Pods across groups of nodes
that share a risk of correlated failure, such as nodes on different racks or
vCenter Servers. Spreading Pods this way helps your workloads stay available
when hardware or software fails. This document explains how topology domains
work and provides guidelines for setting them up.

Setting up a topology domain requires that you enable
[advanced cluster](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/admin-cluster-configuration-file-latest.md.txt#enable-advanced-cluster-field).

## Overview

A *topology domain* is a group of cluster nodes that are considered to be part
of the same logical or physical grouping such as a campus or data center. A
topology domain should correspond to some underlying hardware or software
that has some possibility of correlated failure. For example:

- Software failure such as different vCenter Servers
- Hardware failure such as different racks, different power sources, and different buildings

In Google Distributed Cloud software only for VMware, as part of setting up a topology
domain when you create a cluster, you define a [topology
label](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/reference/vsphere-infrastructure-configuration-file#topology-domains-topology-labels).
After cluster creation, the topology label is populated to labels of nodes in
the topology domain.

To make use of a topology domain, you have the following options:

- Version 1.32 and higher: Configure cluster-level default topology spread
  constraints. For details, see
  [`schedulerConfiguration`](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/user-cluster-configuration-file-latest.md.txt#scheduler-configuration-section).

- Use the Kubernetes cluster-level default constraint,
  `"topology.kubernetes.io/zone"`, as the key in the topology label. For more
  information, see
  [Built-in default constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#internal-default-constraints).

- Configure the `PodTemplate` in your Deployment, StatefulSet, or ReplicaSet,
  as applicable with the topology label key. In the Pod spec, you use the key
  in the topology label as the value for the
  `topologySpreadConstraints.topologyKey` field. This key lets the Kubernetes
  scheduler distribute Pods across the topology domain to ensure high
  availability and prevent over-concentration in any single area in case of
  failure. For more information on configuring `topologySpreadConstraints` in
  your Pod spec, see
  [Pod Topology Spread Constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/)
  in the Kubernetes documentation.

## Example topology domain labels

Suppose you create the following three topology domains when
[creating a user cluster](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/create-user-cluster-topology-domains.md.txt#fill_in_your_vsphere_infrastructure_configuration_file):

    ...
    topologyDomains:
    - name: "topology-domain-1"
      topologyLabels:
        "topology.examplepetstore.com/zone": "zone-1"
    ...

    ...
    topologyDomains:
    - name: "topology-domain-2"
      topologyLabels:
        "topology.examplepetstore.com/zone": "zone-2"
    ...

    ...
    topologyDomains:
    - name: "topology-domain-3"
      topologyLabels:
        "topology.examplepetstore.com/zone": "zone-3"
    ...

After the cluster is created, you update the Pod spec, for example:

    ...
    topologySpreadConstraints:
      topologyKey: "topology.examplepetstore.com/zone"
    ...

At a high level, the Kubernetes scheduler uses
`topology.examplepetstore.com/zone` to separate the cluster nodes into
different groups, `zone-1`, `zone-2`, and `zone-3`. Then the scheduler spreads
the Pods across these three node groups.

## Guidelines for setting up topology domains

To ensure effective use of all cluster resources by the Kubernetes scheduler,
we recommend the following guidelines:

<br />

- The topology domains need to be balanced. You should provide nearly equal amounts of CPU and RAM capacity in each topology domain.
- Provide at least two and preferably three topology domains.
- Don't spread by more than one topology key.
- The nodes should have a similar size in each topology domain.
- If you use [taints and tolerations](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/node-taints.md.txt) for workload separation within a cluster, then each node group should meet the previous requirements.

If these guidelines aren't met, then the scheduler will still try to use the
full capacity of the cluster, but it might take longer to schedule Pods, and not
all Pods will get the expected spreading behavior.

## Limitations

You can't use `gkectl get-config` to
[generate configuration files](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/get-configuration-file-from-cluster.md.txt)
from clusters that use topology domains. This limitation applies to the admin
cluster and to all user clusters that the admin cluster manages. Keep the
configuration files that you use to create and update these clusters, including
the
[vSphere infrastructure configuration file](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/reference/vsphere-infrastructure-configuration-file.md.txt)
and [IP block files](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/ip-block-file.md.txt).
Use these files as the source of truth for your cluster configuration.

## What's next

- [Create an admin cluster for use in topology domains](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/create-admin-cluster-topology-domains.md.txt)
- [Create a user cluster for use in topology domains](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/how-to/create-user-cluster-topology-domains.md.txt)
- [vSphere infrastructure configuration file](https://docs.cloud.google.com/kubernetes-engine/distributed-cloud/vmware/docs/reference/vsphere-infrastructure-configuration-file.md.txt)