Learn how to install Knative serving in your GKE clusters outside Google Cloud. Knative serving now uses Cloud Service Mesh and GKE Enterprise fleets. [Learn what's new and changed with Knative serving
fleet installations](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/install).

<br />

For **Knative serving on VMware** , see the
[GA-level installation guide](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/install/outside-gcp/vmware.md.txt).

> [!WARNING]
>
> **Preview**
>
>
> This product or feature is
>
> subject to the "Pre-GA Offerings Terms" in the General Service Terms section
> of the [Service Specific
> Terms](https://docs.cloud.google.com/terms/service-terms#1).
>
> Pre-GA products and features are available "as is" and might have limited support.
>
> For more information, see the
> [launch stage descriptions](https://cloud.google.com/products/#product-launch-stages).

Support for the following GKE clusters are available as a **Preview**:

- [GKE on AWS](https://docs.cloud.google.com/anthos/clusters/docs/multi-cloud/aws/concepts/architecture)
- [GKE on Azure](https://docs.cloud.google.com/anthos/clusters/docs/multi-cloud/azure/concepts/architecture)
- [Google Distributed Cloud](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/latest/concepts/about-bare-metal)
- [GKE Enterprise attached clusters](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/attached.md.txt)

Use the steps on this page to configure a new installation or to
update an existing [fleet installation](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/install.md.txt)
of Knative serving.

## Before you begin

You must ensure that you meet the
[installation prerequisites](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/install/outside-gcp/prerequisites.md.txt).

## Configure your `CloudRun` custom resource

You need to manually create and configure the `CloudRun` custom resource to
configure your installation.
To manually create a `CloudRun` custom resource for GKE clusters outside Google Cloud:

<br />

1. You must create or have an existing service account that has been
   granted the required
   [Monitoring Metric Writer role](https://docs.cloud.google.com/monitoring/access-control.md.txt#mon_roles_desc)
   (`monitoring.metricWriter`).

   - To create a new service account and then download the key, see
     [Using service accounts](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/securing/service-accounts.md.txt).

   - If you have an existing service account with the necessary permissions,
     locate the key that you downloaded when you created that service account.

2. Ensure Cloud Monitoring is enabled in your GKE cluster:

   - [GKE on AWS](https://docs.cloud.google.com/anthos/clusters/docs/multi-cloud/aws/how-to/create-cluster#telemetry-agent-auth)
   - [GKE on Azure](https://docs.cloud.google.com/anthos/clusters/docs/multi-cloud/azure/how-to/create-cluster#telemetry-agent-auth)
   - [Google Distributed Cloud](https://docs.cloud.google.com/anthos/clusters/docs/bare-metal/latest/how-to/log-monitoring)
   - [GKE Enterprise attached clusters](https://docs.cloud.google.com/kubernetes-engine/multi-cloud/docs/attached.md.txt)
3. Create the `knative-serving` namespace:

       kubectl create namespace knative-serving

4. Create a secret in the `knative-serving` namespace that holds the
   service account with `monitoring.metricWriter` permissions:

       kubectl create secret -n knative-serving generic SECRET_NAME --from-file=PATH_TO_KEY_FILE/SECRET_KEY

   Replace:
   - <var translate="no">SECRET_NAME</var> with the name that you choose for the secret.
   - <var translate="no">SECRET_KEY</var> with the name of the file that includes your credentials. Example: `key.json`
   - <var translate="no">PATH_TO_KEY_FILE</var> with the path to the directory of the <var translate="no">SECRET_KEY</var>.

   See the
   [`kubectl create secret`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#-em-secret-generic-em-) reference to learn more, including optional flags.
5. Create a YAML file with the following attributes, for example
   `cloudrunanthos.yaml`:

   ```yaml
   apiVersion: operator.run.cloud.google.com/v1alpha1
   kind: CloudRun
   metadata:
     name: cloud-run
   spec:
     metricscollector:
       stackdriver:
         projectid: PROJECT_ID
         gcpzone: CLUSTER_LOCATION
         clustername: CLUSTER_NAME
         secretname: SECRET_NAME
         secretkey: SECRET_KEY
   ```


   Replace the following:
   - <var translate="no">PROJECT_ID</var> with the ID of your Google Cloud project.
   - <var translate="no">CLUSTER_LOCATION</var> with the [region or zone](https://docs.cloud.google.com/compute/docs/regions-zones.md.txt#available) in which your cluster is located.
   - <var translate="no">CLUSTER_NAME</var> with the ID of your cluster or the fully qualified identifier for the cluster.
   - <var translate="no">SECRET_NAME</var> with the name of the Secret for the service account of the `knative-serving` namespace.
   - <var translate="no">SECRET_KEY</var> with the key of the Secret for the service account of
     the `knative-serving` namespace. For example: `key.json`

     For details about service accounts, see:
     - [Creating service accounts](https://docs.cloud.google.com/iam/docs/creating-managing-service-accounts)
     - [Creating service account keys](https://docs.cloud.google.com/iam/docs/creating-managing-service-account-keys)

   **Example**

   <br />

   In this example `CloudRun` custom resource, the configuration details for
   Cloud Monitoring are specified to use the `my-gcp-logging-secret` secret
   and `key.json` key:

   ````yaml
    apiVersion: operator.run.cloud.google.com/v1alpha1
    kind: CloudRun
    metadata:
      name: cloud-run
    spec:
      metricscollector:
        stackdriver:
          projectid: my-gcp-project-id
          gcpzone: us-central1-c
          clustername: my-anthos-cluster-name
          secretname: my-gcp-logging-secret
          secretkey: key.json
    ```
   ````

## Enable and install Knative serving

Enable the Knative serving component in your GKE Enterprise fleet
and then deploy your `CloudRun` custom resource:

1. Enable Knative serving in your fleet:

       gcloud container fleet cloudrun enable --project=PROJECT_ID

   Replace <var translate="no">PROJECT_ID</var> with the ID of your Google Cloud project.

   For details and additional options, see the
   [gcloud container fleet cloudrun enable](https://docs.cloud.google.com/sdk/gcloud/reference/container/hub/cloudrun/enable)
   reference.
2. Optional: Verify that the Knative serving feature component is
   enabled:

   ### Console

   View if the Knative serving component is **Enabled** in the
   Google Cloud console:

   [Go to Feature Manager](https://console.cloud.google.com/kubernetes/features)

   ### Command line

   View if the `appdevexperience` state is `ACTIVE`:

   ```console
   gcloud container fleet features list  --project=PROJECT_ID
   ```

   Replace <var translate="no">PROJECT_ID</var> with the ID of your Google Cloud project.

   For details and additional options, see the
   [gcloud container fleet features list](https://docs.cloud.google.com/sdk/gcloud/reference/container/hub/features/list)
   reference.

   Result:

   ```console
   NAME               STATE
   appdevexperience   ACTIVE
   ```
3. For each GKE Enterprise cluster where you want to install Knative serving,

   you must deploy your `CloudRun` custom resource:

   ### gcloud


   ```bash
   gcloud container fleet cloudrun apply --kubeconfig=KUBECONFIG --context=CONTEXT --config=CONFIG_FILE
   ```

   Replace the following:
   - <var translate="no">KUBECONFIG</var> with the absolute path to your kubectl config file.
   - <var translate="no">CONTEXT</var> with the name of the context in your kubectl config file to use for connecting.
   - <var translate="no">CONFIG_FILE</var> with the filename relative path to your `cloudRun` custom resource. Example `cloudrunanthos.yaml`.

   For details and additional options, see the
   [gcloud container fleet cloudrun apply](https://docs.cloud.google.com/sdk/gcloud/reference/container/hub/cloudrun/apply)
   reference.

   <br />

   ### kubectl


   ```bash
   kubectl apply --kubeconfig=KUBECONFIG --filename CONFIG_FILE
   ```

   Replace the following:
   - <var translate="no">KUBECONFIG</var> with the absolute path to your kubectl config file.
   - <var translate="no">CONFIG_FILE</var> with the filename and relative path to your `cloudRun` custom resource. Example `cloudrunanthos.yaml`.

   For details and additional options, see the
   [kubectl apply](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#apply)
   reference.

   <br />


4. Verify that Knative serving has been enabled in your cluster:
   1. Open the Google Cloud console:<br />

      [Go to GKE clusters](https://console.cloud.google.com/kubernetes/list/overview)
   2. Click on the name of your cluster to open the details pane.<br />

      <br />

   3. Run this command to check if you can see the Knative serving version details: `kubectl get all -n knative-serving -o 'go-template={{index .metadata.labels "serving.knative.dev/release"}}'`
   4. Verify that all Knative serving specific deployments are in running status under `knative-serving` and `appdevexperience` namespaces.

## What's next

[Set up your installation of Knative serving](https://docs.cloud.google.com/kubernetes-engine/enterprise/knative-serving/docs/setup.md.txt).