Learn how to install Knative serving in your GKE clusters on Google Cloud. Knative serving now uses Cloud Service Mesh and GKE Enterprise fleets. [Learn what's new and changed with Knative serving
fleet installations](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/install).

<br />

Use this page to customize a new installation or to configure an existing
[fleet installation](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/install.md.txt)
of Knative serving. For example, you can configure for private clusters
or set Cloud Monitoring credentials.

The [default installation steps](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/install.md.txt) automatically
deploy the following `CloudRun` custom resource which enables
metrics on GKE Enterprise on Google Cloud clusters:

```yaml
  apiVersion: operator.run.cloud.google.com/v1alpha1
  kind: CloudRun
  metadata:
    name: cloud-run
```

To learn about custom resources in general, see
[custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/).

## Before you begin

- You must ensure that you meet the
  [installation prerequisites](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/install/on-gcp/prerequisites.md.txt).

- If Knative serving is installed in your cluster, you must ensure that the
  "*GKE add-on* " does not exist before you perform these
  fleet component installation steps. If you have the "*add-on* " installed, you
  must instead [follow the steps for upgrading to GKE Enterprise
  fleets](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/install/on-gcp/upgrade).

  #### Check if the "*GKE add-on*" is
  installed.


  To check if your installation of Knative serving is the
  "*GKE add-on*", run the following command:

  ```bash
  gcloud container clusters describe \
  CLUSTER_NAME \
  --region CLUSTER_LOCATION \
  --project PROJECT_ID --format='get(addonsConfig.cloudRunConfig)'
  ```


  Replace:
  - <var translate="no">CLUSTER_NAME</var> with the name of your cluster.
  - <var translate="no">CLUSTER_LOCATION</var> with the [location](https://docs.cloud.google.com/compute/docs/regions-zones.md.txt#identifying_a_region_or_zone) where your cluster is located.
  - <var translate="no">PROJECT_ID</var> with the ID of your Google Cloud project.


  Results:
  - **"*GKE add-on*" not installed** :
    - Nothing is returned to your terminal if the add-on was never installed.
    - `disabled=true` is returned if the add-on was previously uninstalled.
  - **"*GKE add-on*" is installed** : If the add-on is installed in your cluster, the configuration details for the add-on are returned. Example: `loadBalancerType=LOAD_BALANCER_TYPE_EXTERNAL`

  Example:
  :
      The following example demonstrates that Knative serving was installed in the
      `my-addon-cluster` cluster through the "*GKE add-on*", which
      is configured to handle external traffic:

      ```bash
      gcloud container clusters describe my-addon-cluster \
      --region us-central1-c --project my-gcp-project \
      --format='get(addonsConfig.cloudRunConfig)'
      ```


      Response:

      ```bash
      loadBalancerType=LOAD_BALANCER_TYPE_EXTERNAL
      ```

  <br />

## Configure your `CloudRun` custom resource

You need to manually create and configure the `CloudRun` custom resource to
configure your installation.
To manually create a `CloudRun` custom resource for GKE clusters on Google Cloud:

1. Create a YAML file and include the default configuration, for example
   `cloudrunanthos.yaml`:

   ```yaml
   apiVersion: operator.run.cloud.google.com/v1alpha1
   kind: CloudRun
   metadata:
     name: cloud-run
   ```
2. In `cloudrunanthos.yaml`, you customize the installation by configuring
   one or more of the following attributes under a `spec` section:

   - **Private clusters** : You must add the `clusterConfig` and `isPrivate`
     set of attributes to install Knative serving on a
     [private clusters](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/private-cluster-concept)
     on Google Cloud.

     The `clusterConfig` and `isPrivate` attributes:

     ```yaml
     spec:
       clusterConfig:
         isPrivate: true
     ```

     Note that this setting disables
     [Managed TLS](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/managed-tls.md.txt) because private clusters
     are not able to communicate with the certificate authority by default.
   - **Metrics** : Add the `metricscollector` set of attributes to
     configure Cloud Monitoring. By default, metrics are enable
     for GKE Enterprise on Google Cloud clusters.

     ```yaml
     spec:
       metricscollector:
         stackdriver:
           projectid: PROJECT_ID
           gcpzone: CLUSTER_LOCATION
           clustername: CLUSTER_NAME
           secretname: SECRET_NAME
           secretkey: SECRET_KEY
     ```


     Replace the following:
     - <var translate="no">PROJECT_ID</var> with the ID of your Google Cloud project.
     - <var translate="no">CLUSTER_LOCATION</var> with the [region or zone](https://docs.cloud.google.com/compute/docs/regions-zones.md.txt#available) in which your cluster is located.
     - <var translate="no">CLUSTER_NAME</var> with the ID of your cluster or the fully qualified identifier for the cluster.
     - <var translate="no">SECRET_NAME</var> with the name of the Secret for the service account of the `knative-serving` namespace.
     - <var translate="no">SECRET_KEY</var> with the key of the Secret for the service account of
       the `knative-serving` namespace. For example: `key.json`

       For details about service accounts, see:
       - [Creating service accounts](https://docs.cloud.google.com/iam/docs/creating-managing-service-accounts)
       - [Creating service account keys](https://docs.cloud.google.com/iam/docs/creating-managing-service-account-keys)

     <br />

**Example**

<br />

In this example `CloudRun` custom resource, the configuration details for
Cloud Monitoring are specified to use the `my-gcp-logging-secret` secret
and `key.json` key:

````yaml
 apiVersion: operator.run.cloud.google.com/v1alpha1
 kind: CloudRun
 metadata:
   name: cloud-run
 spec:
   metricscollector:
     stackdriver:
       projectid: my-gcp-project-id
       gcpzone: us-central1-c
       clustername: my-anthos-cluster-name
       secretname: my-gcp-logging-secret
       secretkey: key.json
 ```
````

## Enable and install Knative serving

Enable the Knative serving component in your GKE Enterprise fleet
and then deploy your `CloudRun` custom resource:

1. Enable Knative serving in your fleet:

       gcloud container fleet cloudrun enable --project=PROJECT_ID

   Replace <var translate="no">PROJECT_ID</var> with the ID of your Google Cloud project.

   For details and additional options, see the
   [gcloud container fleet cloudrun enable](https://docs.cloud.google.com/sdk/gcloud/reference/container/hub/cloudrun/enable)
   reference.
2. Optional: Verify that the Knative serving feature component is
   enabled:

   ### Console

   View if the Knative serving component is **Enabled** in the
   Google Cloud console:

   [Go to Feature Manager](https://console.cloud.google.com/kubernetes/features)

   ### Command line

   View if the `appdevexperience` state is `ACTIVE`:

   ```console
   gcloud container fleet features list  --project=PROJECT_ID
   ```

   Replace <var translate="no">PROJECT_ID</var> with the ID of your Google Cloud project.

   For details and additional options, see the
   [gcloud container fleet features list](https://docs.cloud.google.com/sdk/gcloud/reference/container/hub/features/list)
   reference.

   Result:

   ```console
   NAME               STATE
   appdevexperience   ACTIVE
   ```
3. For each GKE Enterprise cluster where you want to install Knative serving,

   you must deploy your `CloudRun` custom resource:

   ### gcloud


   ```bash
   gcloud container fleet cloudrun apply --gke-cluster=CLUSTER_LOCATION/CLUSTER_NAME --config=CONFIG_FILE
   ```

   Replace the following:
   - <var translate="no">CLUSTER_LOCATION</var> with the [region or zone](https://docs.cloud.google.com/compute/docs/regions-zones.md.txt#available) in which your cluster is located.
   - <var translate="no">CLUSTER_NAME</var> with the ID of your cluster or the fully qualified identifier for the cluster.
   - <var translate="no">CONFIG_FILE</var> with the filename relative path to your `cloudRun` custom resource. Example `cloudrunanthos.yaml`.

   For details and additional options, see the
   [gcloud container fleet cloudrun apply](https://docs.cloud.google.com/sdk/gcloud/reference/container/hub/cloudrun/apply)
   reference.

   <br />

   ### kubectl


   ```bash
   kubectl apply --kubeconfig=KUBECONFIG --filename CONFIG_FILE
   ```

   Replace the following:
   - <var translate="no">KUBECONFIG</var> with the absolute path to your kubectl config file.
   - <var translate="no">CONFIG_FILE</var> with the filename and relative path to your `cloudRun` custom resource. Example `cloudrunanthos.yaml`.

   For details and additional options, see the
   [kubectl apply](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#apply)
   reference.

   <br />


4. Verify that Knative serving has been enabled in your cluster:
   1. Open the Google Cloud console:<br />

      [Go to GKE clusters](https://console.cloud.google.com/kubernetes/list/overview)
   2. Click on the name of your cluster to open the details pane.<br />

      <br />

   3. Run this command to check if you can see the Knative serving version details: `kubectl get all -n knative-serving -o 'go-template={{index .metadata.labels "serving.knative.dev/release"}}'`
   4. Verify that all Knative serving specific deployments are in running status under `knative-serving` and `appdevexperience` namespaces.

## What's next

[Set up your installation of Knative serving](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/setup.md.txt).