`MonitoringRule` custom resources in Google Distributed Cloud (GDC) air-gapped let you
derive new time series metrics from existing metrics. Recording rules defined
within a `MonitoringRule` custom resource precompute frequently used or
computationally expensive PromQL expressions and save the results as new
metrics.

Storing precomputed results eliminates redundant calculations each time an
expression is evaluated. This approach reduces query latency for complex
dashboards and alerts that repeatedly evaluate the same metric data.

## Before you begin

To manage `MonitoringRule` custom resources, you must have the necessary permissions.

### Request IAM organization roles

Contact your IAM Org Admin to request the following roles:

- **MonitoringRule PA Creator** (`monitoringrule-pa-creator`): create `MonitoringRule` custom resources.
- **MonitoringRule PA Editor** (`monitoringrule-pa-editor`): edit or delete `MonitoringRule` custom resources.
- **MonitoringRule PA Viewer** (`monitoringrule-pa-viewer`): view `MonitoringRule` custom resources.

### Request IAM project roles

Contact your IAM Project Admin to request the following roles:

- **MonitoringRule Editor** (`monitoringrule-editor`): edit or delete `MonitoringRule` custom resources in the project namespace.
- **MonitoringRule Viewer** (`monitoringrule-viewer`): view `MonitoringRule` custom resources in the project namespace.

## Create recording rules

To create a recording rule, define a name for the recorded metric and a valid
expression that produces a numeric value. This expression, when evaluated,
generates the new metric. Deploy the `MonitoringRule` custom resource in your
project namespace on the Management API server to activate the recording rule.

For more information about recording rules, see
<https://grafana.com/docs/loki/latest/rules/>.

> [!IMPORTANT]
> **Important:** Before creating recording rules, ensure you have a deployed project with active workloads and configured metric scraping. In this way, the rules have data to operate on. For more information, see [Collect metrics](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/gdcag/platform/pa-user/collect-metrics.md.txt).

Follow these steps to create recording rules in your project namespace:

1. Determine the GDC project from which you want to create recording rules.
2. Create a YAML file defining the `MonitoringRule` custom resource.

   [The complete `MonitoringRule` specification](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/gdcag/platform/pa-user/calculate-metrics-from-metrics.md.txt#monitoringrule-recording-rules)
   shows an example for metric-based recording rules. For more information, see
   the [API reference documentation](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/gdcag/apis/service/monitoring/v1/monitoring-v1.md.txt#monitoringrule).
3. Replace the following values in the YAML file according to your needs:

   | Field name | Description |
   |---|---|
   | `namespace` | The project namespace. |
   | `name` | The name for the rule configuration. |
   | `interval` | The duration of the rule evaluation interval in seconds. |
   | `limit` | *Optional.* The maximum number of alerts. Set to `0` for unlimited alerts. |
   | `recordRules` | The definitions for calculating new metrics. |
   | `recordRules.record` | The record name for the new metric. The value must be a valid metric name that defines the time series where the results are stored. |
   | `recordRules.expr` | A [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/) expression for the metric rule, which must evaluate to a numeric value. |
   | `recordRules.labels` | *Optional.* The key-value pairs of labels to add to or overwrite the new metric. |

4. Save the YAML file.

5. Apply the `MonitoringRule` configuration to the Management API server within
   the same namespace as your recording rules:

   ```bash
   kubectl --kubeconfig KUBECONFIG_PATH apply -f MONITORING_RULE_NAME.yaml
   ```

   Replace the following:
   - `KUBECONFIG_PATH`: the path to the kubeconfig file for the Management API server.
   - `MONITORING_RULE_NAME`: the name of the `MonitoringRule` definition file.

## Complete `MonitoringRule` specification

A `MonitoringRule` custom resource contains recording rules that describe the
conditions to create new metrics based on existing metrics for observability.

The following YAML file shows a template for the `MonitoringRule` custom
resource. For more information, see the
[API reference documentation](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/gdcag/apis/service/monitoring/v1/monitoring-v1.md.txt#monitoringrule).

```yaml
# Configures either an alert or a target record for precomputation.
apiVersion: monitoring.gdc.goog/v1
kind: MonitoringRule
metadata:
  # Choose a namespace that matches the project namespace.
  # The alert or record is produced in the same namespace.
  namespace: PROJECT_NAMESPACE
  name: MONITORING_RULE_NAME
spec:
  # Rule evaluation interval.
  interval: 60s

  # Configure the limit for the number of alerts.
  # A value of '0' means no limit.
  # Optional.
  # Default value: '0'
  limit: 0

  # Configure recording rules to generate new metrics based on existing metrics.
  # Recording rules precompute expressions that are frequently needed or computationally expensive.
  # Results are saved as a new set of time series.
  recordRules:
    # Define the time series where you want to write the recording rule.
    # The value must be a valid metric name.
  - record: MyMetricsName

    # Define the PromQL expression to evaluate for this rule.
    expr: rate({service_name="bob-service"} [1m])

    # Define labels to add or overwrite.
    # Map of key-value pairs.
    # Optional.
    labels:
      verb: read
```

Replace the following:

- `PROJECT_NAMESPACE`: your project namespace.
- `MONITORING_RULE_NAME`: the name of the `MonitoringRule` definition file.