<button class="button button-with-icon cloud-select-dropdown__button button-primary" aria-haspopup="true">Version 1.10 (unsupported)</button>

- [Latest](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/latest/installing/creating-clusters/user-cluster-creation)
- [Version 1.16](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.16/installing/creating-clusters/user-cluster-creation.md.txt)
- [Earlier versions (unsupported)](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.10/getting-support.md.txt#version-support)

<br />

In Google Distributed Cloud, user clusters run your workloads, and in a
[multi-cluster architecture](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.10/installing/creating-clusters/create-clusters-overview.md.txt),
user clusters are created and managed by an admin cluster.

Once you've created an admin cluster, calling the `bmctl create config` command
creates a YAML file you can edit to define your user cluster. To apply the
configuration and create the user cluster, use the `bmctl create cluster`
command. Preflight checks are applicable to the user clusters created with
`bmctl create cluster` command.

Keeping workloads off the admin cluster protects sensitive administrative data,
like SSH keys stored in the admin cluster, from those who don't need access
to that information. Additionally, keeping user clusters separate from each
other provides good general security for your workloads.

## Prerequisites

- Latest `bmctl` is downloaded (`gs://anthos-baremetal-release/bmctl/1.10.8/linux-amd64/bmctl`) from Cloud Storage.
- Working admin cluster with access to the cluster API server (the `controlPlaneVIP`).
- Admin cluster nodes have network connectivity to all nodes on the target user cluster.
- Workstation running `bmctl` has network connectivity to all nodes in the target user clusters.
- SSH key used to create user cluster available to root or SUDO user on all nodes in the user cluster.
- Connect-register service account is configured on the admin cluster for use with Connect.

## Enable SELinux

If you want to enable SELinux to secure your containers, you must make sure that
SELinux is enabled in `Enforced` mode on all your host machines. Starting with
Google Distributed Cloud release 1.9.0 or later, you can enable or disable SELinux
before or after cluster creation or cluster upgrades. SELinux is enabled by
default on Red Hat Enterprise Linux (RHEL) and CentOS. If SELinux is disabled on
your host machines or you aren't sure, see
[Securing your containers using SELinux](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.10/installing/configure-selinux.md.txt)
for instructions on how to enable it.

Google Distributed Cloud supports SELinux in only RHEL and CentOS systems.

## Create a user cluster config file

The config file for creating a user cluster is almost exactly the same as the
one used for creating an admin cluster. The only difference is that you remove
the local credentials configuration section to make the config a valid
collection of Kubernetes resources. The configuration section is at the top of
the file under the `bmctl configuration variables` section. For examples of user
cluster configurations, see
[User clusters](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.10/reference/config-samples.md.txt#user_clusters)
in the Cluster configuration samples.

By default, user clusters inherit their credentials from the admin cluster that
manages them. You can selectively override some or all of these credentials.

1. Create a user cluster config file with the `bmctl create config` command:

   bmctl create config -c <var translate="no">USER_CLUSTER_NAME</var>

   For example, issue the following to create a config file for a user cluster
   called `user1`:

   bmctl create config -c user1

   The file is written to `bmctl-workspace/user1/user1.yaml`. The generic path
   to the file is `bmctl-workspace/CLUSTER NAME/CLUSTER_NAME.yaml`
2. Edit the config file with the following changes:

   - Remove the local credentials file paths from the config:

     ```yaml
     ....
       gcrKeyPath: (path to GCR service account key)
       sshPrivateKeyPath: (path to SSH private key, used for node access)
       gkeConnectAgentServiceAccountKeyPath: (path to Connect agent service account key)
       gkeConnectRegisterServiceAccountKeyPath: (path to Hub registration service account key)
       cloudOperationsServiceAccountKeyPath: (path to Cloud Operations service account key)
     ....
     ```
   - Change the config to specify a cluster type of `user` instead of `admin`:

     ```yaml
     ....
     spec:
       # Cluster type. This can be:
       #   1) admin:  to create an admin cluster. This can later be used to create
       #   user clusters.
       #   2) user:   to create a user cluster. Requires an existing admin cluster.
       #   3) hybrid: to create a hybrid cluster that runs admin cluster
       #   components and user workloads.
       #   4) standalone: to create a cluster that manages itself, runs user
       #   workloads, but does not manage other clusters.
       type: user
     ....
     ```
   - Ensure the admin and user cluster specifications for the load balancer VIPs
     and address pools are complementary, and **do not** overlap existing
     clusters. A sample pair of admin and user cluster configurations,
     specifying load balancing and address pools, is shown below:

     ```yaml
     ....
     # Sample admin cluster config for load balancer and address pools
       loadBalancer:
         vips:
           controlPlaneVIP: 10.200.0.49
           ingressVIP: 10.200.0.50
         addressPools:
         - name: pool1
           addresses:
           - 10.200.0.50-10.200.0.70
     ....
     ....
     # Sample user cluster config for load balancer and address pools
     loadBalancer:
         vips:
           controlPlaneVIP: 10.200.0.71
           ingressVIP: 10.200.0.72
         addressPools:
         - name: pool1
           addresses:
           - 10.200.0.72-10.200.0.90
     ....
     ```

     The rest of the user cluster config files are the same as the
     admin cluster config.
   - Specify the pod density of cluster nodes and the container runtime:

     ```yaml
     ....
     # NodeConfig specifies the configuration that applies to all nodes in the cluster.
     nodeConfig:
       # podDensity specifies the pod density configuration.
       podDensity:
         # maxPodsPerNode specifies at most how many pods can be run on a single node.
         maxPodsPerNode: 110
       # containerRuntime specifies which container runtime to use for scheduling containers on nodes.
       # containerd and docker are supported.
       containerRuntime: containerd
     ....
     ```

     For user clusters, allowable values for `maxPodsPerNode` are `32-250`. The
     default value if unspecified is `110`. Once the cluster is created, this
     value cannot be updated.

     The default container runtime is containerd. Alternatively, you can use
     Docker. For more information about changing your runtime,
     see our [Change your container runtime guide](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.10/installing/runtime.md.txt).

     Pod density is also limited by your cluster's available IP resources. For
     details, see
     [Pod networking](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.10/concepts/network-reqs.md.txt#pod_networking).

## Create the user cluster

Issue the `bmctl` command to apply the user cluster config and create the
cluster:

```bash
bmctl create cluster -c USER_CLUSTER_NAME --kubeconfig ADMIN_KUBECONFIG
```

Replace the following:

- <var translate="no">`USER_CLUSTER_NAME`</var>: the cluster name created in the previous section.
- <var translate="no">`ADMIN_KUBECONFIG`</var>: the path to the admin cluster kubeconfig file.

For example, for a user cluster named `user1`, and an admin cluster kubeconfig
file with the path `kubeconfig bmctl-workspace/admin/admin-kubeconfig`, the
command would be:

```bash
bmctl create cluster -c user1 --kubeconfig bmctl-workspace/admin/admin-kubeconfig
```

## Sample user cluster configurations

For example user cluster configurations, see
[User clusters](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.10/reference/config-samples.md.txt#user_clusters) in the
Cluster configuration samples.