To monitor your SAP systems on Google Cloud, you can set up
Cloud Monitoring *alerting policies* that notify your SAP administrators
about potential misconfigurations or resource failures.

This document describes some common HA issues and shows how you can create
alerting policies for them, or use the example alerts as a reference to create
your own custom alerts. The example alerts described in this document use the
Monitoring Query Language (MQL) to query metrics generated by
Google Cloud's Agent for SAP or use Cloud Logging queries to detect host memory
errors. Although by default these alerts apply to all SAP
systems in a given Google Cloud project, you can customize the alerts to filter the
required SIDs or adjust the elapsed time to trigger the alert.

For information about how Cloud Monitoring alerts work, see [Alerting
Overview](https://docs.cloud.google.com/monitoring/alerts).

## Before you begin

- Ensure that you're familiar with the general concepts of
  Monitoring alerting policies. For information about alerting
  policies, see [Alerting overview](https://docs.cloud.google.com/monitoring/alerts.md.txt).

- On each instance that hosts the SAP system that you want to monitor, make sure that Google Cloud's Agent for SAP is installed and [configured to collect the Process Monitoring metrics](https://docs.cloud.google.com/sap/docs/agent-for-sap/latest/configure-process-monitoring.md.txt).

-

  To get the permissions that
  you need to create and modify alerting policies by using the Google Cloud console,

  ask your administrator to grant you the
  following IAM roles on your project:

  - All: [Monitoring Editor](https://docs.cloud.google.com/iam/docs/roles-permissions/monitoring.md.txt#monitoring.editor) (`roles/monitoring.editor`)


  For more information about granting roles, see [Manage access to projects, folders, and organizations](https://docs.cloud.google.com/iam/docs/granting-changing-revoking-access.md.txt).


  You might also be able to get
  the required permissions through [custom
  roles](https://docs.cloud.google.com/iam/docs/creating-custom-roles) or other [predefined
  roles](https://docs.cloud.google.com/iam/docs/roles-overview#predefined).
- To receive the alerts, create the required notification channels.
  For redundancy purposes, we recommend that you create multiple notification
  channels. For more information, see [Create and manage notification
  channels](https://docs.cloud.google.com/monitoring/support/notification-options).

## Import predefined alert policies

Google Cloud provides predefined alert policies that you can import and
set up alerting for some common HA issues. For more information, see the
following sections:

- [Import alerting for location constraint detection](https://docs.cloud.google.com/sap/docs/create-metric-based-alerting.md.txt#location-constraint-detection)
- [Import alerting for resource failure detection](https://docs.cloud.google.com/sap/docs/create-metric-based-alerting.md.txt#resource-failure-detection)
- [Import alerting for SAP HANA replication errors](https://docs.cloud.google.com/sap/docs/create-metric-based-alerting.md.txt#replication-errors)
- [Import alerting for host memory errors](https://docs.cloud.google.com/sap/docs/create-metric-based-alerting.md.txt#host-memory-errors)

### Import alerting for location constraint detection

When you manually move a resource in a Pacemaker cluster using the cluster
commands, then that resource gains a constraint, or client preference is set to
favor a particular node. Such a situation can hinder the resource from failing
over in the event of a system outage. For more information, see
[Moving One Resource](https://clusterlabs.org/pacemaker/doc/2.1/Pacemaker_Administration/html/moving.html#moving-one-resource)
section of the ClusterLabs documentation.

To get notified about such a situation in your SAP HA system running on
Google Cloud, you can import the predefined alert policy
**Pacemaker: Location constraint detected**.

This alert policy notifies when a preference-based constraint is detected and
refers your SAP administrators to the "Unintentional node affinity that favors a
particular node" section of the
[Troubleshooting high-availability configurations for SAP](https://docs.cloud.google.com/sap/docs/troubleshooting/troubleshooting-sap-ha.md.txt#unintentional_node_affinity_that_favors_a_particular_node)
guide. This policy uses the Process Monitoring metric
[`workload.googleapis.com/sap/validation/pacemaker`](https://docs.cloud.google.com/sap/docs/agent-for-sap/latest/process-monitoring.md.txt#process_monitoring_metrics)
which is collected by Google Cloud's Agent for SAP.

To import this alerting policy in your Google Cloud project by using the
Google Cloud console, complete the following steps:


1. In the Google Cloud console, go to the
   ![](https://docs.cloud.google.com/static/monitoring/images/integrations-icon.png) **Integrations**
   page:

   [Go to **Integrations**](https://console.cloud.google.com/monitoring/integrations)

   <br />

   If you use the search bar to find this page, then select the result whose subheading is
   **Monitoring**.
2. Filter for `Google Cloud Agent for SAP`, and then click **View Details**.
3. Navigate to the **Alerts** tab.
4. Select **Pacemaker: Location constraint detected** , and then click **Show Options \> Customise Alert Policy**.
5. (Optional) To configure alerting for one or more specific SAP systems instead of all SAP systems in your Google Cloud project, update the `filter` statement in the **Query editor** as follows:
   1. Remove the `#` character that precedes the `sid` variable.
   2. Specify the required SIDs. To specify multiple SIDs, separate the SIDs using the `|` character. The following is an example of how such a filter statement looks like:

      ```
      sid=~"ABC|HDB|XYZ"
      ```

      In this example, `ABC`, `XYZ`, and
      `HDB` are SIDs.
6. (Optional) Under **Alert Details** , navigate to the **Configure Trigger** . To customize the time that needs to elapse before an alert is triggered, set a value for the **Retest window** field which is long enough to minimize false positives, but short enough to ensure that incidents are opened in a timely manner.
7. Under **Alert Details** , navigate to the **Notifications and name** tab.
8. Select the required notification channels.
9. Review the alert and click **Create Policy**.

<br />

### Import alerting for resource failure detection

In your HA system, if a running resource agent fails, then Pacemaker attempts to
stop that agent and restart it. If the restart operation fails for any reason,
then Pacemaker sets that resource agent's `failcount` value to `INFINITY` (if
`start-failure-is-fatal` is set to `true`, which is the default) and then
attempts to start the agent on a different node. If the resource agent fails to
start on all nodes, then the resource agent remains in the `Stopped` status. To
restore this resource agent back to an operational state, an SAP administrator
must manually clear the resource agent's failcount. For more information on the
failcount behavior of Pacemaker, see the
[ClusterLabs documentation](https://clusterlabs.org/projects/pacemaker/doc/2.1/Pacemaker_Explained/singlehtml/#failure-counts).

To get notified about such a situation in your SAP HA system running on
Google Cloud, you can import the predefined alert policy
**Pacemaker: Resource failed to start**.

This alert policy notifies when a resource agent fails to start and remains in
status `Stopped` for more than 3 minutes. This policy refers your SAP
administrator to the "Resource agent is stopped" section of the
[Troubleshooting high-availability configurations for SAP](https://docs.cloud.google.com/sap/docs/troubleshooting/troubleshooting-sap-ha.md.txt#resource_agent_is_stopped)
guide. This policy uses the Process Monitoring metric
[`workload.googleapis.com/sap/cluster/failcounts`](https://docs.cloud.google.com/sap/docs/agent-for-sap/latest/process-monitoring.md.txt#process_monitoring_metrics),
which is collected by Google Cloud's Agent for SAP.

To import this alerting policy in your Google Cloud project by using the
Google Cloud console, complete the following steps:


1. In the Google Cloud console, go to the
   ![](https://docs.cloud.google.com/static/monitoring/images/integrations-icon.png) **Integrations**
   page:

   [Go to **Integrations**](https://console.cloud.google.com/monitoring/integrations)

   <br />

   If you use the search bar to find this page, then select the result whose subheading is
   **Monitoring**.
2. Filter for `Google Cloud Agent for SAP`, and then click **View Details**.
3. Navigate to the **Alerts** tab.
4. Select **Pacemaker: Resource failed to start** , and then click **Show Options \> Customise Alert Policy**.
5. (Optional) To configure alerting for one or more specific SAP systems instead of all SAP systems in your Google Cloud project, update the `filter` statement in the **Query editor** as follows:
   1. Remove the `#` character that precedes the `sid` variable.
   2. Specify the required SIDs. To specify multiple SIDs, separate the SIDs using the `|` character. The following is an example of how such a filter statement looks like:

      ```
      sid=~"ABC|HDB|XYZ"
      ```

      In this example, `ABC`, `XYZ`, and
      `HDB` are SIDs.
6. (Optional) Under **Alert Details** , navigate to the **Configure Trigger** . To customize the time that needs to elapse before an alert is triggered, set a value for the **Retest window** field which is long enough to minimize false positives, but short enough to ensure that incidents are opened in a timely manner.
7. Under **Alert Details** , navigate to the **Notifications and name** tab.
8. Select the required notification channels.
9. Review the alert and click **Create Policy**.

<br />

### Import alerting for SAP HANA replication errors

In the event of an outage on the SAP HANA primary site, an automated failover from primary to the secondary system isn't possible if the secondary is not in sync with the primary.

This alert policy notifies when the replication status of a highly-available SAP HANA system is not in sync for more than a minute. This policy uses the Process Monitoring metric
[`workload.googleapis.com/sap/hana/ha/replication`](https://docs.cloud.google.com/sap/docs/agent-for-sap/latest/process-monitoring.md.txt#process_monitoring_metrics),
which is derived from [`systemReplication.py`](https://help.sap.com/docs/SAP_HANA_PLATFORM/4e9b18c116aa42fc84c7dbfd02111aba/f6b1bd1020984ee69e902b21b702c096.html) script. This policy refers the SAP administrator to check the status and network connectivity of the primary and secondary SAP HANA systems.

To get notified about such a situation in your SAP HANA HA system running on
Google Cloud, you can import the predefined alert policy
**SAP HANA Replication is not in sync**.

To import this alerting policy in your Google Cloud project by using the
Google Cloud console, complete the following steps:


1. In the Google Cloud console, go to the
   ![](https://docs.cloud.google.com/static/monitoring/images/integrations-icon.png) **Integrations**
   page:

   [Go to **Integrations**](https://console.cloud.google.com/monitoring/integrations)

   <br />

   If you use the search bar to find this page, then select the result whose subheading is
   **Monitoring**.
2. Filter for `Google Cloud Agent for SAP`, and then click **View Details**.
3. Navigate to the **Alerts** tab.
4. Select **SAP HANA Replication is not in sync** , and then click **Show Options \> Customise Alert Policy**.
5. (Optional) To configure alerting for one or more specific SAP systems instead of all SAP systems in your Google Cloud project, update the `filter` statement in the **Query editor** as follows:
   1. Remove the `#` character that precedes the `sid` variable.
   2. Specify the required SIDs. To specify multiple SIDs, separate the SIDs using the `|` character. The following is an example of how such a filter statement looks like:

      ```
      sid=~"ABC|HDB|XYZ"
      ```

      In this example, `ABC`, `XYZ`, and
      `HDB` are SIDs.
6. (Optional) Under **Alert Details** , navigate to the **Configure Trigger** . To customize the time that needs to elapse before an alert is triggered, set a value for the **Retest window** field which is long enough to minimize false positives, but short enough to ensure that incidents are opened in a timely manner.
7. Under **Alert Details** , navigate to the **Notifications and name** tab.
8. Select the required notification channels.
9. Review the alert and click **Create Policy**.

<br />

### Import alerting for host memory errors

When an uncorrectable host memory error occurs on a host running a
Compute Engine instance, Compute Engine automatically
migrates the Compute Engine instance to a healthy host. However, virtual memory pages affected by the
error remain poisoned and inaccessible. If an SAP HANA workload attempts to
read from an affected memory page, the workload immediately fails and crashes.
For more information, see
[Manage host errors for SAP workloads on Google Cloud](https://docs.cloud.google.com/sap/docs/manage-host-errors.md.txt).

To get notified about such a situation on your SAP HANA Compute Engine instances running on
Google Cloud, you can import the predefined alert policy
**Host Memory Alert**.

This alert policy notifies you when an uncorrectable host memory error event is
detected in Cloud Logging. The alert condition matches log entries where the
payload type is `type.googleapis.com/cloud_vmm_proto.HostMemoryAlert`.
The alert notification includes detailed remediation steps, recommending that
you perform a planned failover and a stop and start of the affected instance to
release the poisoned virtual memory pages, as described in
[Protect your SAP workloads from host errors](https://docs.cloud.google.com/sap/docs/manage-host-errors.md.txt#protect_your_sap_workloads_from_host_errors).

To import this alerting policy in your Google Cloud project by using the
Google Cloud console, complete the following steps:

1. In the Google Cloud console, go to the
   ![](https://docs.cloud.google.com/static/monitoring/images/integrations-icon.png) **Integrations**
   page:

   [Go to **Integrations**](https://console.cloud.google.com/monitoring/integrations)

   <br />

   If you use the search bar to find this page, then select the result whose subheading is
   **Monitoring**.
2. Filter for `Google Cloud Agent for SAP`, and then click **View Details**.
3. Navigate to the **Alerts** tab.
4. Select **Host Memory Alert** , and then click **Show Options \> Customise Alert Policy**.
5. (Optional) To configure alerting for one or more specific Compute Engine instances instead of all instances in your Google Cloud project, update the filter in the **Log query** field. For example:

   ```
   resource.labels.instance_id = "INSTANCE_ID"
   ```
6. Under **Alert Details** , navigate to the **Notifications and name** tab.
7. Select the required notification channels.
8. Review the alert and click **Create Policy**.

## Create a custom alerting policy

In addition to importing predefined alert policies, you can update them to
create custom ones that suit your requirements. For this, you can use the
[Google Cloud console](https://docs.cloud.google.com/monitoring/alerts/using-alerting-ui.md.txt), the
[Cloud Monitoring API](https://docs.cloud.google.com/monitoring/api/v3.md.txt), the
[Google Cloud CLI](https://docs.cloud.google.com/sdk/gcloud/reference/alpha/monitoring/policies.md.txt),
or [Terraform](https://cloud.google.com/monitoring/alerts/terraform).

As a starting point, we recommend that you review the
[summary of example alerting policies](https://docs.cloud.google.com/monitoring/alerts/types-of-conditions.md.txt)
as well as the preconfigured alerting policies described in this document.

For information about how to manage or modify alerting policies, see
[Manage alerting policies](https://docs.cloud.google.com/monitoring/alerts/manage-alerts.md.txt).