Configure a cluster to be managed by the GKE On-Prem API

The GKE On-Prem API lets you manage on-premises cluster lifecycles by using the Google Cloud console, the Google Cloud CLI, or Terraform. When you create a cluster with these tools, the API stores cluster metadata in your chosen Google Cloud region. If you create clusters with gkectl, you must enroll them in the GKE On-Prem API to view details and manage their lifecycles through these interfaces.

Enrolling a cluster is separate from registering it to a fleet. All Google Distributed Cloud clusters automatically register to a fleet host project at creation. In the Google Cloud console, registered clusters managed by the GKE On-Prem API show as VMware in the Type column, whereas unenrolled clusters appear as External.

View registered clusters

All your fleet clusters are displayed on the GKE Clusters pages in the console. This both gives you an overview of your entire fleet and, for Google Distributed Cloud, lets you see which clusters are managed by the GKE On-Prem API.

To view your fleet clusters:

In the console, go to the GKE clusters page.
Go to GKE clusters
Select the Google Cloud project.
- If VMware is displayed in the Type column, the cluster is managed by the GKE On-Prem API.
- If External is displayed in the Type column, the cluster isn't managed by the GKE On-Prem API.

Requirements

User clusters must be version 1.11 or higher.
Admin clusters must be version 1.13 or higher.
If you aren't a project owner, minimally, you must be granted the Identity and Access Management role roles/gkeonprem.admin on the project. For details on the permissions included in this role, see GKE on-prem roles in the IAM documentation.

Before you begin

Set up the gcloud CLI, if needed.
1. If you need to install the gcloud CLI, see the gcloud CLI documentation.
2. Update the gcloud CLI components, if needed:
```
gcloud components update
```
Do the following steps to activate and use the GKE On-Prem API:
1. Enable the API in your project:
```
gcloud services enable \
    --project PROJECT_ID \
    gkeonprem.googleapis.com
```
  Replace PROJECT_ID with the project ID of your fleet host project. This is the project ID that was configured in the gkeconnect section of your admin cluster configuration file or user cluster configuration file.
  
  If you get a PERMISSION_DENIED error, double check the project ID that you entered. If the project ID is correct, run gcloud auth login to sign in to the Google Cloud CLI with the account that has access to the project.
2. If this is the first time that you have enabled the GKE On-Prem API in your project, you need to initialize the API. You can do this by calling a gcloud CLI command that displays available versions that you can use to create a cluster:
```
gcloud container vmware clusters query-version-config \
    --project=PROJECT_ID \
    --location=REGION
```
  Replace REGION with us-west1 or another supported region.
If your organization has set up an allowlist that lets traffic from Google APIs and other addresses pass through your proxy server, add the following to the allowlist:
- gkeonprem.googleapis.com
- gkeonprem.mtls.googleapis.com
These are the service names for the GKE On-Prem API.

Enroll a user cluster

gcloud CLI

List the fleet memberships of registered clusters:
```
gcloud container fleet memberships list \
    --project=PROJECT_ID
```
Replace PROJECT_ID with the project ID of your fleet host project.

Note the region in which the admin cluster that manages the user cluster is a member because you will need this in the next step.
Enroll the cluster in the GKE On-Prem API:
```
gcloud container vmware clusters enroll USER_CLUSTER_NAME \
    --project=<var>PROJECT_ID</var> \
    --admin-cluster-membership=projects/PROJECT_ID/locations/MEMBERSHIP_REGION/memberships/ADMIN_CLUSTER_NAME \
    --location=REGION
```
Replace the following:
- USER_CLUSTER_NAME: the name of the user cluster that you want to enroll.
- PROJECT_ID: the project ID of your fleet host project.
- MEMBERSHIP_REGION: the region in which the admin cluster that manages the user cluster is a member. This could be global or a Google Cloud region.
- ADMIN_CLUSTER_NAME: the name of the admin cluster that manages the user cluster.
  
  The ADMIN_CLUSTER_NAME, MEMBERSHIP_REGION, and PROJECT_ID form the fully-specified cluster name for the --admin-cluster-membership flag.
- REGION: the Google Cloud region in which the GKE On-Prem API runs and stores cluster metadata. Specify us-west1 or another supported region. The region can't be changed after the cluster is enrolled.

`gkectl`

Do the following steps on your admin workstation.

Add the following section to the user cluster configuration file:
```
gkeOnPremAPI:
  enabled:true
  location:REGION
```
Replace REGION with the Google Cloud region in which the GKE On-Prem API runs and stores cluster metadata. Specify us-west1 or another supported region. The region can't be changed after the cluster is enrolled.
Update the cluster:
```
gkectl update cluster --kubeconfig ADMIN_CLUSTER_KUBECONFIG --config USER_CLUSTER_CONFIG_FILE
```
Replace the following:
- ADMIN_CLUSTER_KUBECONFIG: the path of the admin cluster kubeconfig file.
- USER_CLUSTER_CONFIG_FILE: the path of your user cluster configuration file.

Enroll an admin cluster

gcloud CLI

List the fleet memberships of registered clusters:
```
gcloud container fleet memberships list \
    --project=PROJECT_ID
```
Replace PROJECT_ID with the project ID of your fleet host project.

Note the region in which the admin cluster is a member because you will need this in the next step.

Enroll the admin cluster in the GKE On-Prem API:

gcloud container vmware admin-clusters enroll ADMIN_CLUSTER_NAME \
    --project=PROJECT_ID \
    --admin-cluster-membership=projects/PROJECT_ID/locations/MEMBERSHIP_REGION/memberships/ADMIN_CLUSTER_NAME \
    --location=REGION

Replace the following:

ADMIN_CLUSTER_NAME: the name of the admin cluster that you want to enroll.
PROJECT_ID the project ID of your fleet host project.
MEMBERSHIP_REGION: the region in which the admin cluster is a member. This could be global or a Google Cloud region.

The ADMIN_CLUSTER_NAME, MEMBERSHIP_REGION, and PROJECT_ID form the fully-specified cluster name for the --admin-cluster-membership flag.
REGION: the Google Cloud region in which the GKE On-Prem API runs and stores cluster metadata.. Specify us-west1 or another supported region. We recommend that you use the same region that is configured in stackdriver.clusterLocation in the admin cluster configuration file. The region can't be changed after the cluster is enrolled.

`gkectl`

Do the following steps on your admin workstation.

Add the following section to the admin cluster configuration file:
```
gkeOnPremAPI:
  enabled:true
  location:REGION
```
Replace REGION with the Google Cloud region in which the GKE On-Prem API runs and stores cluster metadata. Specify us-west1 or another supported region. We recommend that you use the same region that is configured in stackdriver.clusterLocation. The region can't be changed after the cluster is enrolled.
Update the cluster:
```
gkectl update admin --kubeconfig ADMIN_CLUSTER_KUBECONFIG --config ADMIN_CLUSTER_CONFIG_FILE
```
Replace the following:
- ADMIN_CLUSTER_KUBECONFIG: the path of the admin cluster kubeconfig file.
- ADMIN_CLUSTER_CONFIG_FILE: the path of your admin cluster configuration file.

Get information about your cluster

After the cluster is enrolled, you can use the following commands to get information about your clusters:

User cluster

To describe a user cluster:

gcloud container vmware clusters describe USER_CLUSTER_NAME \
  --project=PROJECT_ID \
  --location=REGION

To list your user clusters:

gcloud container vmware clusters list \
  --project=PROJECT_ID \
  --location=-

When you set --location=-, that means to list all clusters in all regions. If you need to scope down the list, set --location to the region you specified when you enrolled the cluster.

Admin cluster

To describe an admin cluster:

gcloud container vmware admin-clusters describe ADMIN_CLUSTER_NAME \
  --project=PROJECT_ID \
  --location=REGION

To list your admin clusters:

gcloud container vmware admin-clusters list \
  --project=PROJECT_ID \
  --location=-

When you set --location=-, that means to list all clusters in all regions. If you need to scope down the list, set --location to the region you specified when you enrolled the cluster.

Connect to the cluster

After the cluster is enrolled with the GKE On-Prem API, you need to choose and configure an authentication method so that you can manage the cluster from the Google Cloud console. The authentication method that you select also controls access to the cluster from the command line. For more information, see the following:

Configure a cluster to be managed by the GKE On-Prem API Stay organized with collections Save and categorize content based on your preferences.

View registered clusters

Requirements

Before you begin

Enroll a user cluster

gcloud CLI

gkectl

Enroll an admin cluster

gcloud CLI

gkectl

Get information about your cluster

User cluster

Admin cluster

Connect to the cluster

Configure a cluster to be managed by the GKE On-Prem API

`gkectl`

`gkectl`