<button class="button button-with-icon cloud-select-dropdown__button button-primary" aria-haspopup="true">Version 1.6 (unsupported)</button>

- [Latest](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/latest/quickstart)
- [Version 1.16](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.16/quickstart.md.txt)
- [Earlier versions (unsupported)](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/getting-support.md.txt#version-support)

<br />

## Introduction to Google Distributed Cloud

<iframe width="560" height="315" src="https://www.youtube.com/embed/RE0A3kHT3LA" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

## Google Distributed Cloud quickstart scope

With Google Distributed Cloud, you can define four types of clusters:

- **admin** - A cluster used to manage user clusters.
- **user** - A cluster used to run workloads.
- **standalone** - A single cluster that can administer itself, and that can also run workloads, but can't create or manage other user clusters.
- **hybrid** - A single cluster for both admin and workloads, that can also manage user clusters.

<br />

In this quickstart, you deploy a **two-node hybrid cluster** with
Google Distributed Cloud. You learn how to create a cluster, and how to monitor
the cluster creation process.

This quickstart assumes a basic understanding of
[Kubernetes](https://kubernetes.io)

## Preparing for Google Distributed Cloud

The install command of Google Distributed Cloud, called `bmctl`, has been designed to
streamline the creation of clusters. The command can automatically set up
the necessary Google service accounts and APIs you need for Google Distributed Cloud,
and we'll use those automated services in this quickstart.

If you want to, you can also manually set up the necessary services and APIs
before you create clusters with `bmctl`. We'll note the command
changes you need to make later in this document, but to set up the Google services
you need manually, see [Enabling Google services and service accounts](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/installing/configure-sa.md.txt).

Before you can create a cluster using Google Distributed Cloud, ensure that you:

1. Create a [Google Cloud project](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects.md.txt) where you have the **Editor** or **Owner** role.
2. Download and install the [`bmctl`
   command-line tool](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/quickstart#installing-bmctl) as described below.
3. Configure a Linux admin workstation for running `bmctl`. Note: Do not use [Cloud Shell](https://docs.cloud.google.com/shell/docs/features) as your admin workstation.
   1. Install `gcloud` and `kubectl` [as described below.](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/quickstart.md.txt#install-gcloud)
   2. Install [Docker version 19.03](https://docs.docker.com/engine/install/) or later. (For instructions on configuring Docker, see the page for configuring your Linux OS: [Configuring CentOS](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/installing/configure-os/centos.md.txt#configuring_docker_1903_on_your_workstation), [Configuring RHEL](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/installing/configure-os/rhel.md.txt#configuring_docker_1903_on_your_workstation), or [Configuring Ubuntu](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/installing/configure-os/ubuntu.md.txt#configuring_docker_1903_on_your_workstation).)
   3. Using `root` access, set up SSH on both the admin workstation and the remote cluster node machines. Initially, you need `root` SSH password authentication enabled on the remote cluster node machines to share keys from the Admin workstation. Once the keys are in place, you can disable SSH password authentication.
   4. Generate a private/public key pair on the admin workstation (do not set a passphrase for the keys). The keys are needed to use SSH for secure, passwordless connections between the administration workstation and the cluster node machines.

      ```
      ssh-keygen -t rsa
      ```


      You can also use `SUDO` user access to the cluster node
      machines to set up SSH, but for passwordless, non-root user connections you
      need to update the `cluster config` YAML file with the
      appropriate credentials. For more information, see the `#Node access
      configuration` section in the
      [sample cluster config
      file.](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/reference/cluster-config).
   5. Add the generated public key to the cluster node machines. By default, the public keys are stored in the `id_rsa.pub` identity file.

      ```
      ssh-copy-id -i ~/.ssh/identity_file root@cluster_node_ip
      ```
   6. Disable SSH password authentication on the cluster node machines and use the following command on the admin workstation to verify the public key authentication works between the admin workstation and the cluster node machines.

      ```
      ssh -o IdentitiesOnly=yes -i identity_file root@cluster_node_ip
      ```

<br />

### Installing gcloud utilities

The `gcloud` and `kubectl` tools are included in
the [gcloud CLI](https://docs.cloud.google.com/sdk).

1. On your administration machine, install and initialize the gcloud CLI using the instructions [here](https://docs.cloud.google.com/sdk/docs.md.txt). This process installs `gcloud`.
2. Update the gcloud CLI:

   ```
   gcloud components update
   ```
3. Log in with your Google account so that you can manage services and service accounts:

   ```
   gcloud auth login --update-adc
   ```

   <br />

   A new browser tab opens and you are prompted to choose an account.

   Note that you can set up a Google Cloud default project at this point,
   and [enable other services and Google APIs](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/installing/configure-sa.md.txt).
   before you create Google Distributed Cloud clusters. Setting a default project
   saves time when you enable services manually.

   However, as shown in this quickstart, you can also specify a project, and
   set up the required Google services, directly with the `bmctl` command when
   you create your clusters. When you do this, `bmctl` always uses the project ID
   you specify when issuing the command.
4. Use `gcloud` to install `kubectl`:

   ```
   gcloud components install kubectl
   ```

   <br />

### Installing bmctl

The command-line tool for creating clusters in Google Distributed Cloud is `bmctl`.
You download `bmctl` from a [Cloud Storage bucket](https://docs.cloud.google.com/storage)

To download `bmctl`:

1. Create a new directory for `bmctl`:

       cd ~
       mkdir baremetal
       cd baremetal

2. Download `bmctl` from the [Cloud
   Storage](https://docs.cloud.google.com/storage) bucket:

       gcloud storage cp gs://anthos-baremetal-release/bmctl/1.6.2/linux-amd64/bmctl bmctl
       chmod a+x bmctl

3. Ensure that `bmctl` is installed correctly by viewing the help information:

   ```
   ./bmctl -h
   ```

<br />

## Creating your cluster nodes

Create two machines that will serve as nodes for your cluster:

- One machine functions as the control plane node.
- One machine functions as the worker node.

Note that your admin workstation needs to be able to `ssh` to these nodes and
have access to the control plane VIP.

See [hardware](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/installing/hardware.md.txt) and operating system
requirements ([Centos](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/installing/configure-os/centos.md.txt),
[RHEL](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/installing/configure-os/rhel.md.txt), and
[Ubuntu](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/installing/configure-os/ubuntu.md.txt)) for more
information on what's required
for the cluster nodes.

## Creating a cluster

To create a cluster:

1. Use `bmctl` to create a config file.
2. Edit the config file to customize it for your cluster and network.
3. Use `bmctl` to create the cluster from the config file.

#### Creating a config file

To create a config file, and enable service accounts and APIs automatically,
make sure you are in the `baremetal` directory, and issue
the `bmctl` command with the following flags:
<br />

```
./bmctl create config -c CLUSTER_NAME \
  --enable-apis --create-service-accounts --project-id=PROJECT_ID
```

<br />

<br />

<var translate="no">CLUSTER_NAME</var> is the name of your cluster, and <var translate="no">PROJECT_ID</var> is a
Google project where you have an **Owner** or **Editor** role.

This command creates a config file under the `baremetal`
directory at: `bmctl-workspace/cluster1/cluster1.yaml`

### Editing the config file

To edit the config file:

1. Open the `bmctl-workspace/cluster1/cluster1.yaml` config file in an editor.
2. Edit the file for your specific node and network requirements. See the sample config file below. Note that in this quickstart, we've omitted the OpenID Connect (OIDC)

```
# gcrKeyPath:  < to GCR service account key>
gcrKeyPath: baremetal/gcr.json
# sshPrivateKeyPath:  < to SSH private key, used for node access>
sshPrivateKeyPath: .ssh/id_rsa
# gkeConnectAgentServiceAccountKeyPath:  < to Connect agent service account key>
gkeConnectAgentServiceAccountKeyPath: baremetal/connect-agent.json
# gkeConnectRegisterServiceAccountKeyPath:  < to Hub registration service account key>
gkeConnectRegisterServiceAccountKeyPath: baremetal/connect-register.json
# cloudOperationsServiceAccountKeyPath:  < to Cloud Operations service account key>
cloudOperationsServiceAccountKeyPath: baremetal/cloud-ops.json
---
apiVersion: v1
kind: Namespace
metadata:
  name: cluster-cluster1
---
apiVersion: baremetal.cluster.gke.io/v1
kind: Cluster
metadata:
  name: cluster1
  namespace: cluster-cluster1
spec:
  # Cluster type. This can be:
  #   1) admin:  to create an admin cluster. This can later be used to create user clusters.
  #   2) user:   to create a user cluster. Requires an existing admin cluster.
  #   3) hybrid: to create a hybrid cluster that runs admin cluster components and user workloads.
  #   4) standalone: to create a cluster that manages itself, runs user workloads, but does not manage other clusters.
  type: hybrid
  # Anthos cluster version.
  anthosBareMetalVersion: 1.6.2
  # GKE connect configuration
  gkeConnect:
    projectID: PROJECT_ID
  # Control plane configuration
  controlPlane:
    nodePoolSpec:
      nodes:
      # Control plane node pools. Typically, this is either a single machine
      # or 3 machines if using a high availability deployment.
      - address:  CONTROL_PLANE_NODE_IP
  # Cluster networking configuration
  clusterNetwork:
    # Pods specify the IP ranges from which Pod networks are allocated.
    pods:
      cidrBlocks:
      - 192.168.0.0/16
    # Services specify the network ranges from which service VIPs are allocated.
    # This can be any RFC 1918 range that does not conflict with any other IP range
    # in the cluster and node pool resources.
    services:
      cidrBlocks:
      - 172.26.232.0/24
  # Load balancer configuration
  loadBalancer:
    # Load balancer mode can be either 'bundled' or 'manual'.
    # In 'bundled' mode a load balancer will be installed on load balancer nodes during cluster creation.
    # In 'manual' mode the cluster relies on a manually-configured external load balancer.
    mode: bundled
    # Load balancer port configuration
    ports:
      # Specifies the port the LB serves the kubernetes control plane on.
      # In 'manual' mode the external load balancer must be listening on this port.
      controlPlaneLBPort: 443
    # There are two load balancer VIPs: one for the control plane and one for the L7 Ingress
    # service. The VIPs must be in the same subnet as the load balancer nodes.
    vips:
      # ControlPlaneVIP specifies the VIP to connect to the Kubernetes API server.
      # This address must not be in the address pools below.
      controlPlaneVIP: CONTROL_PLANE_VIP
      # IngressVIP specifies the VIP shared by all services for ingress traffic.
      # Allowed only in non-admin clusters.
      # This address must be in the address pools below.
      ingressVIP: INGRESS_VIP
    # AddressPools is a list of non-overlapping IP ranges for the data plane load balancer.
    # All addresses must be in the same subnet as the load balancer nodes.
    # Address pool configuration is only valid for 'bundled' LB mode in non-admin clusters.
    # addressPools:
    # - name: pool1
    #   addresses:
    #   # Each address must be either in the CIDR form (1.2.3.0/24)
    #   # or range form (1.2.3.1-1.2.3.5).
    #   - LOAD_BALANCER_ADDRESS_POOL-
    # A load balancer nodepool can be configured to specify nodes used for load balancing.
    # These nodes are part of the kubernetes cluster and run regular workloads as well as load balancers.
    # If the node pool config is absent then the control plane nodes are used.
    # Node pool configuration is only valid for 'bundled' LB mode.
    # nodePoolSpec:
    #   nodes:
    #   - address: LOAD_BALANCER_NODE_IP;
  # Proxy configuration
  # proxy:
  #   url: http://[username:password@]domain
  #   # A list of IPs, hostnames or domains that should not be proxied.
  #   noProxy:
  #   - 127.0.0.1
  #   - localhost
  # Logging and Monitoring
  clusterOperations:
    # Cloud project for logs and metrics.
    projectID: PROJECT_ID
    # Cloud location for logs and metrics.
    location: us-central1
    # Whether collection of application logs/metrics should be enabled (in addition to
    # collection of system logs/metrics which correspond to system components such as
    # Kubernetes control plane or cluster management agents).
    # enableApplication: false
  # Storage configuration
  storage:
    # lvpNodeMounts specifies the config for local PersistentVolumes backed by mounted disks.
    # These disks need to be formatted and mounted by the user, which can be done before or after
    # cluster creation.
    lvpNodeMounts:
      # path specifies the host machine path where mounted disks will be discovered and a local PV
      # will be created for each mount.
      path: /mnt/localpv-disk
      # storageClassName specifies the StorageClass that PVs will be created with. The StorageClass
      # is created during cluster creation.
      storageClassName: local-disks
    # lvpShare specifies the config for local PersistentVolumes backed by subdirectories in a shared filesystem.
    # These subdirectories are automatically created during cluster creation.
    lvpShare:
      # path specifies the host machine path where subdirectories will be created on each host. A local PV
      # will be created for each subdirectory.
      path: /mnt/localpv-share
      # storageClassName specifies the StorageClass that PVs will be created with. The StorageClass
      # is created during cluster creation.
      storageClassName: local-shared
      # numPVUnderSharedPath specifies the number of subdirectories to create under path.
      numPVUnderSharedPath: 5
---
# Node pools for worker nodes
apiVersion: baremetal.cluster.gke.io/v1
kind: NodePool
metadata:
  name: node-pool-1
  namespace: cluster-cluster1
spec:
  clusterName: cluster1
  nodes:
  - address: WORKER_NODE_IP
```

### Running preflight checks and creating the cluster

The `bmctl` command runs preflight checks
on your cluster config file before it creates a cluster. If they're successful,
the cluster is created.

To run preflight checks and create the cluster:

1. Ensure that you are in the `baremetal` directory.
2. Use the following command to create the cluster:

```
./bmctl create cluster -c CLUSTER_NAME
```
For example:

```
./bmctl create cluster -c cluster1
```
3. The` bmctl `command monitors the preflight checks and cluster creation, and then displays output to the screen and writes verbose information to the `bmctl` logs.

<br />

The `bmctl `logs, as well as the preflight check and node installation logs,
are in the directory:
`baremetal/bmctl-workspace/CLUSTER_NAME/log`

The `bmctl` preflight checks the proposed cluster installation for the
following conditions:

<br />

- The Linux distribution and version are supported.
- SELinux is not in "enforcing" mode.
- For Ubuntu, AppArmor and UFW are not active.
- For CentOS/RHEL, firewalld is not active.
- Google Container Registry is reachable.
- The VIPs are available.
- The cluster machines have connectivity to each other.
- Load balancer machines are on the same L2 subnet.

<br />

It can take several minutes to finish the cluster creation.

### Getting information about your cluster

After the cluster is created successfully, the `kubectl` command
shows you information about the new cluster. During cluster creation, the
`bmctl` command writes a kubeconfig file for the cluster that you
query with `kubectl`. The kubeconfig file is written to `bmctl-workspace/CLUSTER_NAME/CLUSTER_NAME-kubeconfig`.

For example:

```
kubectl --kubeconfig bmctl-workspace/cluster1/cluster1-kubeconfig get nodes
```

<br />

This command returns:

```
NAME      STATUS   ROLES    AGE   VERSION
node-01   Ready    master   16h   v1.17.8-gke.16
node-02   Ready    <none>   16h   v1.17.8-gke.16
```

<br />

If your cluster creation fails preflight checks, then check the preflight check logs for
errors, and correct them in the cluster config file. The preflight check logs are located
in the `/log`directory at

```
~/baremetal/bmctl-workspace/CLUSTER_NAME/log
```

<br />

The preflight check logs for each machine in the cluster are in the
<var translate="no">CLUSTER_NAME</var> directory, and are organized by IP address.
For example:

```
bmctl-workspace/cluster1/log
└── preflight-20201007-034844
    ├── 172.17.0.3
    ├── 172.17.0.4
    ├── 172.17.0.5
    ├── 172.17.0.6
    ├── 172.17.0.7
    └── node-network
```

<br />

### Ignoring pre-flight check errors

If your cluster creation fails after pre-flight checks, you can try to re-install the
cluster by using the `--force `flag in the` bmctl `command.

The `--force ` flag installs over an existing cluster, but ignores
the results from any preflight check failure due to already allocated server ports.

1. Ensure that you are in the `baremetal` directory.
2. Use the following command with the `--force `flag to re-create the cluster:

```
./bmctl create cluster -c CLUSTER_NAME --force
```
For example:

```
./bmctl create cluster -c cluster1 --force
```

## Creating a Deployment and a Service

Here's a manifest for a Deployment:

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  selector:
    matchLabels:
      app: metrics
      department: sales
  replicas: 3
  template:
    metadata:
      labels:
        app: metrics
        department: sales
    spec:
      containers:
      - name: hello
        image: "gcr.io/google-samples/hello-app:2.0"
```

<br />

Save the manifest as `my-deployment.yaml`, and create the Deployment:
<br />

```
kubectl --kubeconfig bmctl-workspace/cluster1/cluster1-kubeconfig create -f my-deployment.yaml
```

<br />

<br />

View the Deployment:
<br />

```
kubectl --kubeconfig bmctl-workspace/cluster1/cluster1-kubeconfig get deployments
```

<br />

<br />

The output shows that the Deployment has three running Pods:

```
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
my-deployment      3/3     3            3           16s
```

<br />

Here's a manifest for a Service of type LoadBalancer:

```
apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: metrics
    department: sales
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 8080
```

<br />

Save the manifest as `my-service.yaml`, and create the Service:
<br />

```
kubectl --kubeconfig bmctl-workspace/cluster1/cluster1-kubeconfig create -f my-service.yaml
```

<br />

<br />

View the Service:
<br />

```
kubectl --kubeconfig bmctl-workspace/cluster1/cluster1-kubeconfig get service my-service
```

<br />

<br />

Output:

```
NAME         TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S
my-service   LoadBalancer   172.26.232.2   172.16.1.21   80:30060/TCP
```

<br />

Notice that Google Distributed Cloud has given the service an external IP address.
Use the external IP address to call the Service:

```
curl 172.16.1.21
```

<br />

The output is a hello world message:

```
Hello, world!
Version: 2.0.0
Hostname: my-deployment-75d45b64f9-6clxj
```

<br />

## Creating a high availability control plane

This quickstart creates a simple two-node hybrid cluster. If you want to create a high availability
control plane, create a cluster that has three control plane nodes.

For example, edit the config file shown above to add to additional nodes to the control
plane:

```
controlPlane:
  nodePoolSpec:
    clusterName: cluster1
    nodes:
    # Control Plane node pools. Typically, this is either a single machine
    # or 3 machines if using a high availability deployment.
    - address: <Machine 1 IP>
    - address: <Machine 2 IP>
    - address: <Machine 3 IP>
```

<br />

## Running the load balancer in its own node pool

This quickstart creates a simple two-node hybrid cluster. The load balancer runs on the same
node that runs the control plane.

If you want the load balancer to run in its own node pool, edit the `nodePoolSpec`
values of the `loadBalancer` section of your [config file](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/1.6/quickstart.md.txt#edit-config):

```
  loadBalancer:
    nodePoolSpec:
      clusterName: "cluster1"
      nodes:
      - address: <LB Machine 1 IP>
      - address: <LB Machine 2 IP>
```