This page describes how to upgrade the HPE Chassis Manager firmware to version `2.10.12` or later when `gdcloud appliance upgrade` stops at the `ChassisFirmware` step during an [appliance update and patch](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/admin/update-patch.md.txt#start-the-upgrade).

## Before you begin

Before initiating the firmware upgrade, complete the following prerequisites:

1. Run the upgrade command by following the steps in [Update and patch device](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/admin/update-patch.md.txt#start-the-upgrade):

   ```bash
   gdcloud appliance upgrade
   ```
2. Wait for the command to pause at the `ChassisFirmware` step. If the command detects a Chassis Manager firmware version earlier than `2.10.12`, it automatically extracts the required firmware update ISO from the release bundle and saves it to your local file system before stopping.

3. **Ensure power redundancy**: connect and verify both power cables to the appliance chassis power supplies to prevent accidental power loss during firmware flashing.

   > [!CAUTION]
   > **Caution:** To prevent Chassis Manager firmware corruption, connect both power supply cables to the chassis for power redundancy and don't interrupt power during the upgrade process. A power loss while flashing eMMC partitions can corrupt the firmware.

4. **Verify network access**: ensure the upgrade machine can reach the compute blade Integrated Lights-Out (iLO) and Chassis Manager IP addresses on the management network:

   - **Node 1 (bm01) iLO** : `https://198.18.0.7`
   - **Node 2 (bm02) iLO** : `https://198.18.0.8`
   - **Node 3 (bm03) iLO** : `https://198.18.0.9`
   - **Chassis Manager** : `https://198.18.0.2`

   If the endpoints are unreachable, ensure the upgrade machine is configured with a static IP address on the management subnet (see [Connect the laptop to the switch](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/admin/connect-the-device.md.txt)) or see [Troubleshoot appliance access issues](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/infrastructure/io-troubleshooting/appliance-inaccessibility.md.txt).
5. **Locate the extracted HPE hybrid update ISO** : verify that `gdcloud appliance upgrade` extracted the firmware ISO in your unpacked release directory:

   ```bash
   ls -lh /root/gdch/full-release-VERSION/images/gpc-system-firmware/EL8000CM_Updater_BI_2.10.12_App_2.10.12.hybrid.iso
   ```

   Replace `VERSION` with the target patch version (for example, `1.14.4-gdch.0`).

   If you run your web browser on a separate workstation instead of the upgrade machine, copy `EL8000CM_Updater_BI_2.10.12_App_2.10.12.hybrid.iso` to that workstation so you can select it in the iLO file dialog.
6. **Retrieve blade iLO and Chassis Manager credentials** : get the administrator credentials for a compute blade iLO (`bmc-credentials-*`) and the Chassis Manager (`chassis-*-credential`) from the `gpc-system` namespace:

   ```bash
   # List the BMC and Chassis credential secrets
   kubectl --kubeconfig=$KUBECONFIG get secrets -n gpc-system | grep -E 'bmc-credentials|chassis-.*-credential'

   # Retrieve the blade iLO username and password
   kubectl --kubeconfig=$KUBECONFIG get secret -n gpc-system BMC_CREDENTIAL_SECRET -o jsonpath='{.data.username}' | base64 -d; echo
   kubectl --kubeconfig=$KUBECONFIG get secret -n gpc-system BMC_CREDENTIAL_SECRET -o jsonpath='{.data.password}' | base64 -d; echo

   # Retrieve the existing Chassis Manager username and password
   kubectl --kubeconfig=$KUBECONFIG get secret -n gpc-system CHASSIS_CREDENTIAL_SECRET -o jsonpath='{.data.username}' | base64 -d; echo
   kubectl --kubeconfig=$KUBECONFIG get secret -n gpc-system CHASSIS_CREDENTIAL_SECRET -o jsonpath='{.data.password}' | base64 -d; echo
   ```

   Replace the following:
   - `BMC_CREDENTIAL_SECRET`: the name of a `bmc-credentials-*` secret listed by the `kubectl get secrets` command.
   - `CHASSIS_CREDENTIAL_SECRET`: the name of the `chassis-*-credential` secret listed by the `kubectl get secrets` command.

## Upgrade the Chassis Manager firmware

Follow these steps to mount the extracted ISO in iLO and flash the Chassis Manager firmware from a compute blade:

### Connect to blade iLO HTML5 console

1. Open a web browser and go to any compute blade iLO (for example, `https://198.18.0.8`).
2. Sign in using the blade iLO administrator credentials retrieved in [Before you begin](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/admin/chassis-firmware-upgrade.md.txt#before-you-begin).
3. From the navigation menu, click **Remote Console \& Media** and launch the **HTML5 Remote Console**.

   > [!NOTE]
   > **Note:** If the remote console is unavailable, go to the **Access Settings** page to enable the remote console feature before proceeding.

### Disable UEFI Secure Boot

The HPE Chassis Manager firmware update ISO is an unsigned live Linux environment. To prevent a `Verification failed: (0x1A) Security Violation` error when booting the ISO, disable UEFI Secure Boot:

1. In the HTML5 Remote Console, click **Power** \> **Reset** to restart the compute blade.
2. During POST on the **HPE Edgeline** splash screen, press **F9** to enter **System Utilities**.

   > [!NOTE]
   > **Note:** If your keyboard doesn't send function keys (`F9` or `F10`) to the remote console, click **Virtual Keyboard** in the HTML5 Remote Console menu bar to send the keys. If you miss the POST prompt, in the iLO web interface go to **Virtual Media** \> **Boot Order** , set **One-Time Boot Option** to **RBSU (BIOS Setup)** , click **Apply** , and then reset the server from **Power \& Thermal** \> **Server Power** \> **Reset**.

3. Select **System Configuration** \> **BIOS/Platform Configuration (RBSU)** \> **Server Security** \> **Secure Boot Settings**.

4. Set **Attempt Secure Boot** to **Disabled**.

5. Press **F10** to save the configuration.

6. Stay in **System Utilities** while you mount the update ISO in the next section.

### Mount the extracted HPE hybrid update ISO

1. In the HTML5 Remote Console window, click **Virtual Media** in the menu bar.
2. Select **CD/DVD** \> **Local \*.iso File**.
3. Click **Browse** and select the extracted `EL8000CM_Updater_BI_2.10.12_App_2.10.12.hybrid.iso` file.
4. Ensure **Boot on Next Reset** is selected.
5. Click **Insert Media** and verify that the virtual media status shows **Connected**.

   > [!IMPORTANT]
   > **Important:** Keep the HTML5 Remote Console window open until the firmware update completes. Closing the console window disconnects the local ISO file stream.

### Restart server blade and run the firmware update

1. Exit **System Utilities** to restart the compute blade.
2. When prompted by the GNU GRUB bootloader menu in the HTML5 console, select **Live system (amd64)** and press **Enter** to start the firmware update.

   > [!NOTE]
   > **Note:** To prevent corrupting the firmware during the 3-to-5-minute partition flashing process, don't interrupt power or close the console window. Update progress appears live in the console.

### Re-enable Secure Boot and reboot server blade

1. When the installation completion screen appears, click **Virtual Media** in the HTML5 Remote Console menu bar and eject the CD/DVD media.
2. Enter `reboot` in the console to restart the server blade.
3. During POST on the **HPE Edgeline** splash screen, press **F9** to enter **System Utilities**.
4. Select **System Configuration** \> **BIOS/Platform Configuration (RBSU)** \> **Server Security** \> **Secure Boot Settings** , and set **Attempt Secure Boot** to **Enabled**.
5. Press **F10** to save the configuration, and then exit **System Utilities** so the blade boots back into the node operating system.

### Reconfigure Chassis Manager network and administrator credentials

Flashing the firmware reimages the Chassis Manager eMMC storage and resets settings to factory defaults. Unlike hardware bootstrapping (where a local DHCP server assigns `198.18.0.2`), deployed appliances don't run DHCP on the management network, so IPv4 DHCP makes `198.18.0.2` unreachable until you restore the static IPv4 configuration. Flashing also resets the `Administrator` password to `password` and removes custom user accounts.

Complete the following steps to restore network connectivity and credentials:

1. Wait 2 to 3 minutes for the Chassis Manager internal ARM processor to finish initializing.
2. From the upgrade machine connected to the appliance management interface, find the Chassis Manager link-local IPv6 address:

   ```bash
   ip -6 neigh show dev INTERFACE
   ```

   Replace `INTERFACE` with the management network interface name on the upgrade machine (for example, `mgmt`).

   Look for the entry matching the Chassis Manager MAC address (for example, `fe80::5eed:8cff:fe61:c206`).
3. Connect to the Chassis Manager over SSH using its link-local IPv6 address and interface name:

   ```bash
   ssh "Administrator@CHASSIS_IPV6_LINK_LOCAL%INTERFACE"
   ```

   Replace `CHASSIS_IPV6_LINK_LOCAL` with the link-local IPv6 address of the Chassis Manager from the previous step.
4. When prompted for a password, enter the factory default password: `password`.

5. When the system prompts you to change the initial `Administrator` password, enter and confirm the password retrieved from your `chassis-*-credential` Kubernetes secret in [Before you begin](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/admin/chassis-firmware-upgrade.md.txt#before-you-begin). After changing the password, the Chassis Manager closes the initial SSH session.

6. Reconnect over SSH using the new `Administrator` password to open the `EL8000CM->` shell:

   ```bash
   ssh "Administrator@CHASSIS_IPV6_LINK_LOCAL%INTERFACE"
   ```
7. In the `EL8000CM->` shell, restore the static IPv4 configuration:

   ```text
   set network 198.18.0.2 255.255.255.224 198.18.0.1
   ```
8. Enable switch link propagation:

   ```text
   enable linkpropagation
   ```
9. If your `chassis-*-credential` Kubernetes secret specifies the `gdch` username instead of `Administrator`, recreate the `gdch` administrator account in the `EL8000CM->` shell:

   ```text
   add user gdch PASSWORD admin
   ```

   Replace `PASSWORD` with the password from your `chassis-*-credential` Kubernetes secret.
10. Enter `exit` to close the `EL8000CM->` shell, and then verify from the upgrade machine that `198.18.0.2` responds:

    ```bash
    ping -c 3 198.18.0.2
    ```

### Resume the appliance upgrade

After the Chassis Manager is online and reports version `2.10.12` or later, resume `gdcloud appliance upgrade`.

Because `PreflightCheckAppliance`, `PreflightCheckHardware`, and `SetupRegistry` completed before stopping at `ChassisFirmware`, you can resume the remaining steps directly without rerunning the registry loading process:

```bash
cd /root/gdch/full-release-VERSION/release

./gdcloud appliance upgrade --step=ChassisFirmware && \
./gdcloud appliance upgrade --step=Organization && \
./gdcloud appliance upgrade --step=Storage && \
./gdcloud appliance upgrade --step=Cleanup
```

Alternatively, if you want to run the full upgrade sequence from the beginning, run `./gdcloud appliance upgrade` without `--step`.