This page shows how to create a user cluster for Google Distributed Cloud.

The instructions here are complete. For a shorter introduction to creating a
user cluster, see
[Create a user cluster (quickstart)](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-quickstart).

## Before you begin

[Create an admin cluster](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/create-admin-cluster).

# Get an SSH connection to your admin workstation

[Get an SSH connection](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/create-admin-workstation#getting_an_ssh_connection_to_your_admin_workstation)
to your admin workstation.

Recall that `gkeadm` activated your
component access service account on the admin workstation.

Do all the remaining steps in this topic on your admin workstation in the home
directory.

## Credentials configuration file

When you used `gkeadm` to create your admin workstation, you filled in a
credentials configuration file named `credential.yaml`. This file holds the
username and password for your vCenter server.

## User cluster configuration file

When `gkeadm` created your admin workstation, it generated a configuration file
named `user-cluster.yaml`. This configuration file is for creating your user
cluster.

## Filling in your configuration file

### `name`

Set the
[`name`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#name-field)
field to a name of your choice for the user cluster.

### `gkeOnPremVersion`

This field is already filled in for you.

### `vCenter`

The values you set in the `vCenter` section of your
[admin cluster configuration file](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/admin-cluster-configuration-file)
are global. That is, they apply to your admin cluster and your user clusters.

For each user cluster that you create, you have the option of overriding some of
the global `vCenter` values.

If you want to override any of the global `vCenter` values, fill in the relevant
fields in the
[`vCenter`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#vcenter-section)
section of your user cluster configuration file.

### `network`

Set `network.ipMode.type` to the same value that you set in your
[admin cluster configuration file](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/admin-cluster-configuration-file):
either `"dhcp"` or `"static"`.

If you set `ipMode.type` to `"static"`, create an
[IP block file](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/ip-block-file)
that provides the static IP addresses for the nodes in your user cluster. Then
set
[`network.ipMode.ipBlockFilePath`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#network-ipmode.ipblockfilepath-field)
to the path of your IP block file.

Provide values for the remaining fields in the
[`network`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#network-section)
section.

Regardless of whether you rely on a DHCP server or specify a list of static IP
addresses, you need enough IP addresses to satisfy the following:

- The nodes in your user cluster

- An additional node in the user cluster to be used temporarily during upgrades

As mentioned previously, if you want to use static IP addresses, then you need
to provide an IP block file. Here is an example of an IP block file with six
hosts. This is enough addresses for a cluster that has five nodes and an
occasional sixth node for upgrades:

```yaml
blocks:
  - netmask: 255.255.252.0
    gateway: 172.16.23.254
    ips:
    - ip: 172.16.20.21
      hostname: user-host1
    - ip: 172.16.20.22
      hostname: user-host2
    - ip: 172.16.20.23
      hostname: user-host3
    - ip: 172.16.20.24
      hostname: user-host4
    - ip: 172.16.20.25
      hostname: user-host5
    - ip: 172.16.20.26
      hostname: user-host6
```

### `loadBalancer`

Set aside a VIP for the Kubernetes API server of your user cluster. Set aside
another VIP for the ingress service of your user cluster. Provide your VIPs as
values for
[`loadBalancer.vips.controlPlaneVIP`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#loadbalancer-vips-controlplanevip-field)
and
[`loadBalancer.vips.ingressVIP`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#loadbalancer-vips-ingressvip-field).

Set `loadBalancer.kind` to the same value that you set in your
[admin cluster configuration file](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/admin-cluster-configuration-file):
`"ManualLB"`, `"F5BigIP"`, or `"Seesaw"`. Then fill in the corresponding
section:
[`manualLB`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#loadbalancer-manuallb-section),
[`f5BigIP`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#loadbalancer-f5bigip-section),
or
[`seesaw`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#loadbalancer-seesaw-section).

### `masterNode`

Fill in the
[`masterNode`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#masternode-section)
section.

### `nodePools`

Fill in the
[`nodePools`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#nodepools-section)
section.

### `antiAffinityGroups`

Set
[`antiAffinityGroups.enabled`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#antiaffinitygroups-enabled-field)
to `true` or `false`.

### `authentication`

If you want to use OpenID Connect (OIDC) to authenticate users, fill in the
[`authentication.oidc`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#authentication-oidc-section)
section.

If you want to provide an additional serving certificate for your user cluster's
vCenter server, fill in the
[`authentication.sni`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#authentication-sni-section)
section.

### `stackdriver`

Fill in the
[`stackdriver`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#stackdriver-section)
section.

### `gkeConnect`

Fill in the
[`gkeConnect`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#gkeconnect-section)
section.

### `usageMetering`

If you want to enable usage metering for your cluster, then fill in the
[`usageMetering`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#usagemetering-section)
section.

### `cloudAuditLogging`

If you want to integrate the audit logs from your cluster's Kubernetes API
server with Cloud Audit Logs, fill in the
[`cloudAuditLogging`](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/user-cluster-configuration-file#cloudauditlogging-section)
section.

## Validating your configuration file

After you've filled in your user cluster configuration file, run
`gkectl check-config` to verify that the file is valid:

```
gkectl check-config --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] --config [CONFIG_PATH]
```

where:

- <var translate="no">\[ADMIN_CLUSTER_KUBECONFIG\]</var> is the path of the kubeconfig file for
  your admin cluster.

- <var translate="no">\[CONFIG_PATH\]</var> is the path of your user cluster configuration
  file.

If the command returns any failure messages, fix the issues and validate the
file again.

If you want to skip the more time-consuming validations, pass the `--fast` flag.
To skip individual validations, use the `--skip-validation-xxx` flags. To
learn more about the `check-config` command, see
[Running preflight checks](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/preflight-checks).

## Creating a Seesaw load balancer for your user cluster

If you have chosen to use the bundled Seesaw load balancer, do the step in this
section. Otherwise, skip this section.

Create and configure the VMs for your
[Seesaw load balancer](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/bundled-load-balance):

```
gkectl create loadbalancer --kubeconfig ADMIN_CLUSTER_KUBECONFIG --config user-cluster.yaml
```

where

- <var translate="no">ADMIN_CLUSTER_KUBECONFIG</var> is the path of the kubeconfig file for your admin cluster.

## Upload Container-Optimized OS to vCenter

If you have chosen to create a Container-Optimized OS nodepool for the user cluster, then you must upload Container-Optimized OS to vCenter.

To upload Container-Optimized OS to vCenter, run this command:

```
gkectl prepare --config ADMIN_CLUSTER_CONFIG_PATH --extra-os-image-types cos
```

where <var translate="no">ADMIN_CLUSTER_CONFIG_PATH</var> is the path of your admin cluster configuration file.

## Creating a user cluster

Create a user cluster:

```
gkectl create cluster --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] \
  --config [CONFIG_PATH]
```

where

- <var translate="no">\[CONFIG_PATH\]</var> is the path of your user cluster configuration
  file.

- <var translate="no">\[ADMIN_CLUSTER_KUBECONFIG\]</var> is the path of the kubeconfig file for
  your admin cluster.

The `gkectl create cluster` command creates a kubeconfig file named
`[USER_CLUSTER_NAME]-kubeconfig` in the current directory. You will need this
kubeconfig file later to interact with your user cluster.

## Locate the user cluster kubeconfig file

The `gkectl create cluster` command creates a kubeconfig file named
`USER_CLUSTER_NAME-kubeconfig` in the current directory. You will need this
kubeconfig file later to interact with your user cluster.

If you like, you can change the name and location of your kubeconfig file.

## Verify that your user cluster is running

Verify that your user cluster is running:

```
kubectl get nodes --kubeconfig USER_CLUSTER_KUBECONFIG
```

Replace <var translate="no">USER_CLUSTER_KUBECONFIG</var> with the path of your user cluster
kubeconfig file.

The output shows the user cluster nodes. For example:

```
my-user-cluster-node-pool-69-d46d77885-7b7tx   Ready ...
my-user-cluster-node-pool-69-d46d77885-lsvzk   Ready ...
my-user-cluster-node-pool-69-d46d77885-sswjk   Ready ...
```

## Troubleshooting

See [Troubleshooting cluster creation and upgrade](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/troubleshoot-cluster-creation-upgrade).

## What's next

- [Connect to your cluster from the Google Cloud console](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/connecting-to-a-cluster).
- [Deploy an application](https://docs.cloud.google.com/anthos/clusters/docs/on-prem/1.8/how-to/deploy-first-app).