> [!CAUTION]
> You are currently viewing version 1.3 of the Apigee hybrid documentation. **This version is end of life.** You should upgrade to a newer version. For more information, see [Supported versions](https://docs.cloud.google.com/apigee/docs/hybrid/supported-platforms.md.txt#supported-versions).

<button class="button button-with-icon cloud-select-dropdown__button button-primary" aria-haspopup="true">Version 1.3 (unsupported)</button>

- Supported versions:
- [v1.17 (latest)](https://docs.cloud.google.com/apigee/docs/hybrid/v1.17/upgrade.md.txt)
- [v1.16](https://docs.cloud.google.com/apigee/docs/hybrid/v1.16/upgrade.md.txt)
- [v1.15](https://docs.cloud.google.com/apigee/docs/hybrid/v1.15/upgrade.md.txt)
- [v1.14](https://docs.cloud.google.com/apigee/docs/hybrid/v1.14/upgrade.md.txt)
- [List of supported versions](https://docs.cloud.google.com/apigee/docs/hybrid/supported-platforms.md.txt#supported-versions)
- Unsupported versions:
- [v1.13](https://docs.cloud.google.com/apigee/docs/hybrid/v1.13/upgrade.md.txt)
- [v1.12](https://docs.cloud.google.com/apigee/docs/hybrid/v1.12/upgrade.md.txt)
- [v1.11](https://docs.cloud.google.com/apigee/docs/hybrid/v1.11/upgrade.md.txt)
- [v1.10](https://docs.cloud.google.com/apigee/docs/hybrid/v1.10/upgrade.md.txt)
- [v1.9](https://docs.cloud.google.com/apigee/docs/hybrid/v1.9/upgrade.md.txt)
- [v1.8](https://docs.cloud.google.com/apigee/docs/hybrid/v1.8/upgrade.md.txt)
- [v1.7](https://docs.cloud.google.com/apigee/docs/hybrid/v1.7/upgrade.md.txt)
- [v1.6](https://docs.cloud.google.com/apigee/docs/hybrid/v1.6/upgrade.md.txt)
- [v1.5](https://docs.cloud.google.com/apigee/docs/hybrid/v1.5/upgrade.md.txt)
- [v1.4](https://docs.cloud.google.com/apigee/docs/hybrid/v1.4/upgrade.md.txt)
- [v1.3](https://docs.cloud.google.com/apigee/docs/hybrid/v1.3/upgrade.md.txt)
- [v1.2](https://docs.cloud.google.com/apigee/docs/hybrid/v1.2/upgrade.md.txt)
- [v1.1](https://docs.cloud.google.com/apigee/docs/hybrid/v1.1/upgrade.md.txt)

> [!NOTE]
> **Note:** This procedure covers upgrading from Apigee hybrid version 1.2.0 to Apigee hybrid version 1.3.6 only.
>
>
> If you are upgrading from Apigee hybrid version 1.0 or
> 1.1, you must first upgrade to hybrid version 1.2 before upgrading to version 1.3.6.
> See the instructions for [Upgrading
> Apigee hybrid to version 1.2](https://docs.cloud.google.com/apigee/docs/hybrid/v1.2/upgrade).

## Upgrading to version 1.3.6 overview.

> [!WARNING]
> Upgrading to Apigee hybrid version 1.3.6 requires downtime during the
>
>
> To minimize downtime in production hybrid environments, make sure you are running at least
> two clusters, and update the clusters one at a time.
>
>
> In addition, upon upgrade each component will perform a rolling restart.


The procedures for upgrading Apigee hybrid are organized in the following sections:

1. Preparation
   1. Create and update service accounts.
   2. Plan environment groups.
   3. Copy and update overrides file.
2. Upgrade Istio and cert-manager.
3. Install hybrid runtime version 1.3.
4. Clean up.

> [!NOTE]
> **Note:**Do not create new environments during the upgrade process.

## Prerequisite

- Apigee hybrid version 1.2. If you are updating from an earlier version see the instructions for [Upgrading Apigee hybrid to version 1.2](https://docs.cloud.google.com/apigee/docs/hybrid/v1.2/upgrade.md.txt).

## Preparation

1. (Recommended) Make a backup copy of your version 1.2 `$APIGEECTL_HOME/` directory. For example:

       tar -czvf $APIGEECTL_HOME/../apigeectl-v1.2-backup.tar.gz $APIGEECTL_HOME

2. (Recommended) Backup your Cassandra database following the instructions in [Cassandra backup and recovery](https://docs.cloud.google.com/apigee/docs/hybrid/v1.3/backup-recovery.md.txt)
3. Upgrade your Kubernetes platform as follows. Follow your platform's documentation if you need help:

   | Platform | Upgrade to version |
   |---|---|
   | GKE | 1.15.x |
   | Anthos | 1.5 |
   | AKS | 1.16.x using [Anthos attached clusters](https://cloud.google.com/anthos/docs/setup/attach-kubernetes-clusters) |

4. If you are not using Apigee Connect in your hybrid installation, enable Apigee Connect.
   1. Check to see if the Apigee Connect API is enabled:

      ```
      gcloud services list | grep apigeeconnect

      apigeeconnect.googleapis.com         Apigee Connect API
      ```
   2. If it is not, enable the API:

      ```
      gcloud services enable apigeeconnect.googleapis.com --project $PROJECT_ID
      ```


      Where **$PROJECT_ID** is your Google Cloud project ID.
   3. On the command line, get your `gcloud` authentication credentials, as the following example shows:

      ```
      TOKEN=$(gcloud auth print-access-token)
      ```

      To check that your token was populated, use `echo`, as the following example shows:

      ```
      echo $TOKEN
      ```

      This should display your token as an encoded string.

      For more information, see
      [gcloud command-line tool overview](https://cloud.google.com/sdk/gcloud/).
   4. Check to see if Apigee Connect is enabled for your organization:

      ```
      curl -H "Authorization: Bearer $TOKEN" \
        "https://apigee.googleapis.com/v1/organizations/$ORG_NAME"
      ```


      Where **$ORG_NAME** is the ID of your organization.


      If the output contains:

      ```
            "name" : "features.mart.connect.enabled",
            "value" : "true"
      ```


      Apigee Connect is enabled.
   5. If Apigee Connect is not enabled, assign the Apigee Connect Agent role to the MART service account:

          gcloud projects add-iam-policy-binding $PROJECT_ID \
            --member serviceAccount:apigee-mart@$PROJECT_ID.iam.gserviceaccount.com \
            --role roles/apigeeconnect.Agent

      > [!TIP]
      > **Tip:** If you need to get the email address of your MART service account, run the following command:
      >
      > ```
      > gcloud iam service-accounts list | grep mart
      > ```

   6. Enable Apigee Connect with the following command:

      ```
      curl -H "Authorization: Bearer $TOKEN" -X PUT \
        -H "Content-Type: application/json" \
        -d '{
          "name" : "'"$ORG_NAME"'",
          "properties" : {
            "property" : [ {
              "name" : "features.hybrid.enabled",
              "value" : "true"
            }, {
              "name" : "features.mart.connect.enabled",
              "value" : "true"
            } ]
          }
        }' \
        "https://apigee.googleapis.com/v1/organizations/$ORG_NAME"
      ```


      If the output contains the two following properties, Apigee Connect was successfully enabled:

      ```
            {
              "name": "features.mart.connect.enabled",
              "value": "true"
            },
            {
              "name": "features.hybrid.enabled",
              "value": "true"
            }
      ```
5. Create the **`apigee-watcher`** service account. Apigee Watcher is a new service account introduced in v1.3. It whaches the synchronizer for org-level changes and applies those changes to configure the Istio ingress.


   From your main hybrid directory:

   ```
   ./tools/create-service-account apigee-watcher ./service-accounts
   ```
6. Assign Apigee Runtime Agent role to the Watcher service account:

   ```
   gcloud projects add-iam-policy-binding $PROJECT_ID \
     --member serviceAccount:apigee-watcher@$PROJECT_ID.iam.gserviceaccount.com \
     --role roles/apigee.runtimeAgent
   ```


   Where **`PROJECT_ID`** is your Google Cloud project ID. If your service
   account email addresses differ from this pattern, replace them accordingly.

   > [!TIP]
   > **Tip:** If you need to get the email address of your watcher service account, run the following command:
   >
   > ```
   > gcloud iam service-accounts list | grep watcher
   > ```


   The output should include a list of all the service accounts and their roles, including:

   ```
     ...
   - members:
     - serviceAccount:apigee-watcher@hybrid13rc5.iam.gserviceaccount.com
     role: roles/apigee.runtimeAgent
     ...
   ```
7. Plan your [environment groups](https://docs.cloud.google.com/apigee/docs/hybrid/v1.3/environment-groups-about) for routing. Apigee hybrid 1.3 manages base path routing with environment groups instead of `routingRules`. If you are using `routingRules` in your hybrid configuration, design environment groups to replicate your routing.


   You must create at least one environment group.


   See [About environment groups](https://docs.cloud.google.com/apigee/docs/hybrid/v1.3/environment-groups-about).

   ### *Click to see an example of using environment groups to replace
   `routingRules`*

   In this example, the following `routingRules` in the `overrides.yaml` file
   route requests to the *foo* and *bar* API proxies to the *prod-env-1*
   environment, and the *foo-test* proxy to the *test-env* environment:

   ```
   virtualhosts:
     - name: default
       hostAliases:
         - "*.acme.com"
       sslCertPath: ./certs/keystore.pem
       sslKeyPath: ./certs/keystore.key
       routingRules:
         - paths:
           - /foo
           - /bar
         - env: prod-env-1
         - paths:
           - foo-test
         -env: test-env
   ```


   To duplicate that in hybrid version 1.3.6:
   1. Deploy the *foo* and *bar* proxies to the *prod-env-1* environment.
   2. Deploy the *foo-test* and *bar* proxy to the *test-env* environment.
   3. Create an environment group with a hostalias for the proxies, like `api.acme.com`.

      > [!TIP]
      > **Tip:** Create an environment group with the same name as the virtualhosts name you used in version 1.2.

   4. Revise the `virtualhosts` stanza in `overrides.yaml` to
      - Replace `virtualhosts:name` with the name of the environment group.
      - Remove the `virtualhosts:routingRules`.


      For example:

      ```
      virtualhosts:
        - name: default # environment group name
          sslCertPath: ./certs/keystore.pem
          sslKeyPath: ./certs/keystore.key
      ```
8. Update your overrides file:
   1. Make a copy of your overrides file.
   2. Update **gcp** and **k8sCluster** stanzas.


      The following configuration properties have been replaced in hybrid version 1.3:
      - `gcpRegion` replaced with `gcp:region`
      - `gcpProjectID` replaced with `gcp:projectID`
      - `gcpProjectIDRuntime` replaced with `gcp:gcpProjectIDRuntime`
      - `k8sClusterName` replaced with `k8s:clusterName`
      - `k8sClusterRegion` replaced with `k8s:clusterRegion`


      For example, replace the following structure:

      ```
      gcpRegion: gcp region
      gcpProjectID: gcp project ID
      gcpProjectIDRuntime: gcp project ID

      k8sClusterName: name
      k8sClusterRegion: region
      ```


      with:

      ```
      gcp:
       projectID: gcp project ID
       region: gcp region
       gcpProjectIDRuntime: gcp project ID # optional. This is only required if you
                                             # want logger/metrics data to be sent in
                                             # different gcp project.

      k8sCluster:
       name: gcp project ID
       region: gcp region
      ```
   3. If you do not have a unique instance identifier already in your overrides file, add one:

      ```
      # unique identifier for this installation. 63 chars length limit
      instanceID: ID
      ```


      Where <var translate="no">ID</var> is a unique identifier for this hybrid installation, like
      "`my-hybrid-131-installation`" or "`acmecorp-hybrid-131`."
   4. Add the Watcher (`apigee-watcher`) service account to the overrides file:

      ```
      # Note: the SA should have the "Apigee Runtime Agent" role
      watcher:
       serviceAccountPath: "service account file"
      ```
   5. Add the Metrics (`apigee-metrics`) service account to the overrides file:

      ```
      metrics:
       serviceAccountPath: "service account file"
      ```
   6. Update `virtualhosts:` stanza to replace `routingRules`with your environment group.
      1. `-name: `Replace the name with the name of your environment group. You can have multiple name entries, one for each environment group.
      2. `hostAliases:[]` Delete this line.
      3. Keep (or add) the `sslCertPath:` and `sslKeyPath:` entries.
      4. Delete all `routingRules` entries.


      For example:

      ```
      virtualhosts:
        - name: default
          hostAliases:
            - "*.acme.com"
          sslCertPath: ./certs/keystore.pem
          sslKeyPath: ./certs/keystore.key
          routingRules:
            - paths:
              - /foo
              - /bar
            - env: my-environment
      ```


      Becomes:

      ```
      virtualhosts:
        - name: example-env-group
          sslCertPath: ./certs/keystore.pem
          sslKeyPath: ./certs/keystore.key
      ```
   7. Update the `mart` and `connectAgent:` stanzas.

      > [!NOTE]
      > **Note:** If you are not planning to use Apigee Connect in your hybrid 1.3 installation, skip this step and follow the **tip** in step 5 in [Apigee Connect](https://docs.cloud.google.com/apigee/docs/hybrid/v1.3/apigee-connect.md.txt#add-apigee-connect-to-the-hybrid-runtime) after upgrading.

      1. Under `mart:` remove the `hostAlias:`, `sslCertPath:`, and `sslKeyPath:` entries.
      2. Add an `connectAgent:` stanza.
      3. Under `connectAgent:` add a `serviceAccountPath:` entry and provide the path to the service acount file that has the Apigee Connect Agent role assigned to it (usually the MART service account).


      For example:

      ```
      mart:
        hostAlias: "mart.apigee-hybrid-docs.net"
        serviceAccountPath: ./service-accounts/hybrid-project-apigee-mart.json
        sslCertPath: ./certs/fullchain.pem
        sslKeyPath: ./certs/privkey.key
      ```


      Becomes:

      ```
      mart:
        serviceAccountPath: ./service-accounts/hybrid-project-apigee-mart.json

      connectAgent:
        serviceAccountPath: ./service-accounts/hybrid-project-apigee-mart.json
      ```

## Upgrade Istio and cert-manager


Apigee hybrid version 1.3 requires cert-manager v0.14.2 to manage and verify certificates and
the Istio distribution provided with Anthos Service Mesh (ASM) version 1.5.7 (or newer) to create and manage
the runtime ingress gateway.

### Upgrade Istio 1.4.6 to ASM 1.5.7 (or newer)

> [!WARNING]
> Traffic disruption may occur during the upgrade process. To minimize the disruption, ensure that at least two replicas of each component (except Citadel) are running.

1. To minimize downtime, the Istio deployments and HPAs must have at least two replicas each. Run the following commands to determine the number of replicas:

       kubectl -n istio-system get deployments # list of deployments
       kubectl -n istio-system get hpa # list of hpa

2. Edit each deployment that has only one replica and increase the `replicas:` to `2` or more:

       kubectl -n istio-system edit deployment name


   For example:

   ```
   spec:
     progressDeadlineSeconds: 600
     replicas: 2
   ```
3. Edit each HPA that has only one replica and increase the `minReplicas:` to `2` or more:

       kubectl -n istio-system edit hpa name


   For example:

   ```
   spec:
     maxReplicas: 5
     minReplicas: 2
   ```
4. Download and install ASM following the install instructions in [Download and install
   ASM](https://docs.cloud.google.com/apigee/docs/hybrid/v1.3/install-download-cert-manager-istio#download-and-install-asm).
5. After the installation, run the version command to make sure you have 1.5.x installed correctly:

   ```
   ./bin/istioctl version

   client version: 1.5.8-asm.0
   apigee-mart-ingressgateway version:
   citadel version: 1.4.6
   galley version: 1.4.6
   ingressgateway version: 1.5.8-asm.0
   pilot version: 1.4.6
   policy version: 1.4.6
   sidecar-injector version: 1.4.6
   telemetry version: 1.4.6
   pilot version: 1.5.8-asm.0
   data plane version: 1.4.6 (1 proxies), 1.5.8-asm.0 (2 proxies)
   ```

### Upgrade cert-manager

> [!NOTE]
> Apigee hybrid version 1.3 requires cert-manager version 0.14.2. You can follow the complete instructions on the cert-manager site for [Upgrading
> from v0.13 to v0.14](https://cert-manager.io/docs/installation/upgrading/upgrading-0.13-0.14/). The steps needed for hybrid are provided below.

1. Delete the current cert-manager deployment:

   ```
   kubectl delete -n cert-manager deployment cert-manager cert-manager-cainjector cert-manager-webhook
   ```
2. Verify your version of Kubernetes:

   ```
   kubectl version
   ```
3. Run the following command to install cert-manager from Jetstack:

   ```
   kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.14.2/cert-manager.yaml 
   ```

## Install the hybrid runtime

1. Store the latest version number in a variable:

   ```
   export VERSION=$(curl -s \
       https://storage.googleapis.com/apigee-public/apigee-hybrid-setup/current-version.txt?ignoreCache=1)
   ```
2. Check that the variable was populated with a version number. If you want to use a different version, you can save that in environment variable instead. For example:

   ```
   echo $VERSION
     1.3.6
   ```
3. Download the release package for your operating system:

   **Mac 64 bit:**

   ```
   curl -LO \
       https://storage.googleapis.com/apigee-public/apigee-hybrid-setup/$VERSION/apigeectl_mac_64.tar.gz
   ```

   **Linux 64 bit:**

   ```
   curl -LO \
       https://storage.googleapis.com/apigee-public/apigee-hybrid-setup/$VERSION/apigeectl_linux_64.tar.gz
   ```

   **Mac 32 bit:**

   ```
   curl -LO \
       https://storage.googleapis.com/apigee-public/apigee-hybrid-setup/$VERSION/apigeectl_mac_32.tar.gz
   ```

   **Linux 32 bit:**

   ```
   curl -LO \
       https://storage.googleapis.com/apigee-public/apigee-hybrid-setup/$VERSION/apigeectl_linux_32.tar.gz
   ```
4. Rename your current `apigeectl/` directory to a backup directory name. For example:

       mv $APIGEECTL_HOME/ $APIGEECTL_HOME-v1.2/ 

5. Extract the downloaded gzip file contents into your hybrid base directory. For example:

   ```
   tar xvzf filename.tar.gz -C hybrid-base-directory
   ```
6. `cd` to the base directory.
7. The tar contents are, by default, expanded into a directory with the version and platform in its
   name. For example: `./apigeectl_1.0.0-f7b96a8_linux_64`. Rename that directory
   to `apigeectl`:

       mv apigeectl_1.0.0-f7b96a8_linux_64 apigeectl

8. Delete the `apigee-resources-install` job from `apigee-system`:

   ```
   kubectl -n apigee-system delete job apigee-resources-install
   ```
9. Delete the older CRD:

   ```
   kubectl delete crd apigeetelemetries.apigee.cloud.google.com
   ```
10. Update the `cassandra:` stanza in your overrides file with an `externalSeedHost` property. This property will help ensure your new hybrid version 1.3.6 installation will use the same Kubernetes cluster as your version 1.2 installation. This is a one-time step needed only for upgrade from hybrid version 1.2 to version 1.3.6 (or newer).
    1. Find one of the IP addresses of the existing Cassandra in the same Kubernetes cluster where you are upgrading the 1.2.0 installation.

       ```
       kubectl -n namespace get pods -o wide
       ```


       Where <var translate="no">namespace</var> is your Apigee hybrid namespace.


       Make note of the IP address of a Cassandra node. For example:

       ```
       kubectl -n apigee get pods -o wide
       NAME                  READY   STATUS    RESTARTS   AGE   IP          NODE
       apigee-cassandra-0    1/1     Running   0          33d   10.68.8.24   gke-example-cluster-rc5-apigee-data-c8bf1234-09kc
       apigee-cassandra-1    1/1     Running   0          16d   10.68.8.33   gke-example-cluster-rc5-apigee-data-c9221ee7-10kc
       apigee-cassandra-2    1/1     Running   0          23h   10.68.9.11   gke-example-cluster-rc5-apigee-data-d123e456-11kc
       ```
    2. Add the value for the `externalSeedHost` property:

       ```
       cassandra:
        externalSeedHost: Cassandra_node_IP
       ```


       Where <var translate="no">Cassandra_node_IP</var> is the IP of the cassandra node
       (**`10.68.8.24`** in the preceding example).

       > [!NOTE]
       > **Note:** You will remove the `externalSeedHost` property in a later step after your new Cassandra is running and verified.

11. In the **new** `apigeectl/` directory, run `apigeectl init`, `apigeectl apply`, and `apigeectl check-ready`:
    1. Initialize hybrid 1.3.6:

       ```
       apigeectl init -f overrides_1.3.yaml
       ```


       Where <var translate="no">overrides_1.3.yaml</var> is your edited overrides.yaml file.
    2. In hybrid version 1.3, the syntax of the `--dry-run` flag depends on the version of `kubectl` you are running. Check the version of `kubectl`:

       ```
       gcloud version
       ```
    3. Check for errors with a dry run:
       `kubectl` version 1.17 and older:

       ```
       apigeectl apply -f overrides_1.3.yaml --dry-run=true
       ```


       `kubectl` version 1.18 and newer:

       ```
       apigeectl apply -f overrides_1.3.yaml --dry-run=client
       ```
    4. Apply your overrides. Select and follow the instructions for production environments or demo/experimental environments, depending on your installation.

       > [!TIP]
       > During the upgrade process each component will perform a rolling restart. Therefore, for production environments, it is best to apply the upgrade to one component at a time.

       ### Production


       For production environments you should upgrade each hybrid component individually, and
       check the status of the upgraded component before proceding to the next component.
       1. Apply your overrides to upgrade Cassandra:

          ```
          apigeectl apply -f overrides_1.3.yaml --datastore
          ```
       2. Check completion:

          ```
          kubectl -n namespace get pods
          ```


          Where <var translate="no">namespace</var> is your Apigee hybrid namespace.


          Proceed to the next step only when the pods are ready.
       3. Apply your overrides to upgrade Telemetry components and check completion:

          ```
          apigeectl apply -f overrides_1.3.yaml --telemetry
          ```

          ```
          kubectl -n namespace get pods
          ```
       4. Apply your overrides to upgrade the org-level components (MART, Watcher and Apigee Connect) and check completion:

          ```
          apigeectl apply -f overrides_1.3.yaml --org
          ```

          ```
          kubectl -n namespace get pods
          ```
       5. Apply your overrides to upgrade your environments. You have two choices:
          - Apply your overrides to one environment at a time and check completion. Repeat this step for each environment:

            ```
            apigeectl apply -f overrides_1.3.yaml --env env_name
            ```

            ```
            kubectl -n namespace get pods
            ```


            Where <var translate="no">env_name</var> is the name of the environment you are upgrading.
          - Apply your overrides to all environments at once and check completion:

            ```
            apigeectl apply -f overrides_1.3.yaml --all-envs
            ```

            ```
            kubectl -n namespace get pods
            ```

       ### Demo/Experimental


       In most demo or experimental environments, you can apply the overrides to all components
       at once. If your demo/experimental environment large and complex or closely mimics a
       production environment, you may want to use the instructions for upgrading
       [production environments](https://docs.cloud.google.com/apigee/docs/hybrid/v1.3/upgrade.md.txt#production)
       1.

          ```
          apigeectl apply -f overrides_1.3.yaml
          ```
       2. Check the status:

          ```
          apigeectl check-ready -f overrides_1.3.yaml
          ```


       For more instructions see
       [GKE Hybrid setup - Step 5: Install hybrid on GKE](https://docs.cloud.google.com/apigee/docs/hybrid/v1.3/install-apply-hybrid.md.txt).
    5. Once you have hybrid 1.3 setup up and running, verify that all of the Cassandra nodes (old and new) are part of the same Cassandra cluster. Run the following command on one of the Cassandra nodes:

           kubectl -n namespace get pods
           kubectl -n namespace exec old Cassandra pod -- nodetool status


       In the following example output, 10.68.8.24 is from version 1.2.0 and is the node IP you
       used as the `externalSeedHost`. 10.68.7.11 is from version 1.3.6:

       ```
       Datacenter: dc-1
       ================
       Status=Up/Down
       |/ State=Normal/Leaving/Joining/Moving
       --  Address     Load        Tokens       Owns (effective)  Host ID                               Rack
       UN  10.68.8.24  379.41 KiB  256          50.8%             11bbd43b-af64-464b-a96d-0d6dd0521de1  ra-1
       UN  10.68.7.11  1.35 MiB    256          49.2%             0b4d9e08-f353-413a-b4a9-7d18a8d07e58  ra-1
       ```


       If they are not in the same cluster, [check the
       `externalSeedHost` value](https://docs.cloud.google.com/apigee/docs/hybrid/v1.3/upgrade#add-externalseedhost).
    6. Once all the pods are up and running, remove `externalSeedHost` from the your overrides file and run `apigeectl apply` again with `--datastore` option:

       ```
       apigeectl apply --datastore -f overrides_1.3.6.yaml
       ```

    ## Clean up


    Once you have verified all the pods are up and healthy and the ASM endpoints are valid for the
    new installation, you can clean up:
    - Hybrid 1.2 resources.
    - The older Cassandra instance
    - Istio 1.4.6 resources.

    ### Delete Hybrid 1.2.0 resources

    > [!NOTE]
    > **Note:** This procedure requires the `apigeectl` version 1.2 command in your backed up directory. If you need to download another copy of `apigeectl` version 1.2, you can use the following command:
    >
    > ```
    > curl -LO \
    >     https://storage.googleapis.com/apigee-public/apigee-hybrid-setup/1.2.0/apigeectl_your_system.tar.gz
    > ```
    >
    >
    > Where <var translate="no">your_system</var> is one of `linux_32`, `linux_64`,
    > `mac_32`, or `mac_64`.

    1. Remove the 1.2.0 virtualhost routing details:

       ```
       $APIGEECTL_HOME-v1.2/apigeectl delete -s virtualhost -f 1.2.0_overrides.yaml
       ```


       Where <var translate="no">$APIGEECTL_HOME-v1.2</var> is the directory where you backed up your
       `apigeectl` version 1.2 directory.
    2. If the endpoint is still working as expected and you have verified all the 1.3.0 components are working run the following command to delete the hybrid 1.2.0 resources:

       ```
       $APIGEECTL_HOME-v1.2/apigeectl delete -c "mart,connect-agent,synchronizer,runtime,udca,metrics,logger" \
         -f 1.2.0_overrides.yaml
       ```

    ### Decommission the older Cassandra instance

    1. `cd` into the newly installed `apigeectl`directory.
    2. Run the `tools/cas_cleanup.sh` script.


       This script decommissions the old Cassandra pod from the Cassandra ring, deletes old STS, and deletes PVCs.

       ```
       bash cas_cleanup.sh Apigee namespace
       ```

    ### Delete Istio version 1.4.6 resources

    > [!NOTE]
    > **Note:** Deleting the Istio Istio version 1.4.6 resources will also delete the MART ingress as well. MART ingress is replaced with Apigee Connect in Apigee hybrid version 1.3.

    1. Run the following command to delete the most recent Istio v.1.4.6 resources:

       ```
       kubectl delete all -n istio-system --selector \
         'app in (apigee-mart-istio-ingressgateway, galley, security, istio-nodeagent, istio-mixer, sidecarInjectorWebhook, istio-mixer)'
       ```
    2. Run the following commands to delete older jobs from the Istio 1.4.6 installation:

           kubectl -n istio-system delete job istio-init-crd-10-1.4.6
           kubectl -n istio-system delete job istio-init-crd-11-1.4.6
           kubectl -n istio-system delete job istio-init-crd-14-1.4.6


    **Congratulations!** You have successfully upgraded to Apigee hybrid version 1.3.6.