Cassandra graceful shutdown

You're viewing Apigee and Apigee hybrid documentation.
View Apigee Edge documentation.

To ensure data integrity during maintenance, you must shut down the Cassandra pod gracefully before rescheduling the pod.

Before you begin

Before proceeding with any maintenance that involves restarting Cassandra pods, ensure the current cluster is healthy.

Check the status of the Cassandra cluster:

kubectl -n APIGEE_NAMESPACE -c apigee-cassandra exec apigee-cassandra-default-0 -- bash -c 'nodetool -u cassandra -pw $CASS_PASSWORD status'

Sample output:

Datacenter: us-west2
====================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address     Load      Tokens  Owns (effective)  Host ID                               Rack
UN  10.44.16.4  5.43 MiB  256     100.0%            c873909a-3003-4283-b175-92289967a486  ra-1
UN  10.44.14.4  5.45 MiB  256     100.0%            8126b7a0-a175-475b-b9be-43b1f9b78487  ra-1
UN  10.44.18.4  5.42 MiB  256     100.0%            6655989c-1313-4df2-92af-c131c0f223c0  ra-1

All the nodes in the cluster should be in the UN (Up/Normal) state.

Graceful shutdown

To gracefully shut down the Cassandra pod, you can either drain the node or delete the pod.

Drain the node to trigger a graceful shutdown of the Cassandra pod. This action evicts the pod from the current node, and the scheduler attempts to move the pod on another available node.
- If a node is available, the pod restarts on that node.
- If no other node is available, the pod remains in a Pending state until you add a new node to the cluster.
After the pod restarts, it automatically updates its IP address within the cluster.

Note: This step is only recommended when node requires maintenance.
To force the Cassandra pod to gracefully shut down, you can use the following command to delete the pod:
```
kubectl delete pod TARGET_CASSANDRA_POD -n APIGEE_NAMESPACE
```
This command deletes the Cassandra pod, triggering its graceful shutdown process.

Verification

To confirm that the Cassandra node has shut down gracefully, inspect the logs of the Cassandra container.

kubectl logs TARGET_CASSANDRA_POD -n APIGEE_NAMESPACE -c apigee-cassandra

Where TARGET_CASSANDRA_POD is the name of the Cassandra pod that you are checking.

The log entries should indicate successful drain and flush.