The monitoring component of Google Distributed Cloud (GDC) air-gapped lets you encrypt and
authenticate application metrics in transit by using mutual Transport Layer
Security (mTLS). Configuring mTLS on a `MonitoringTarget` custom resource
verifies the identity of your metrics server and monitoring pipeline to secure
metrics collection across project workloads.

## Before you begin

To manage `MonitoringTarget` custom resources, you must have the necessary permissions.

### Request IAM organization roles

Contact your IAM Org Admin to request the following roles:

- **MonitoringTarget PA Creator** (`monitoringtarget-pa-creator`): create `MonitoringTarget` custom resources.
- **MonitoringTarget PA Editor** (`monitoringtarget-pa-editor`): edit or modify `MonitoringTarget` custom resources.
- **MonitoringTarget PA Viewer** (`monitoringtarget-pa-viewer`): view `MonitoringTarget` custom resources.

### Request IAM project roles

Contact your IAM Project Admin to request the following roles:

- **MonitoringTarget Editor** (`monitoringtarget-editor`): edit or modify `MonitoringTarget` custom resources in the project namespace.
- **MonitoringTarget Viewer** (`monitoringtarget-viewer`): view `MonitoringTarget` custom resources in the project namespace.

After obtaining the necessary permissions, complete these steps before enabling
metrics encryption:

1. Deploy your metrics server.
2. [Collect metrics from your workloads](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/gdcag/application/ao-user/collect-metrics).
3. Ensure your `MonitoringTarget` custom resource shows a `Ready` status, like
   in the following example:

   ```yaml
   apiVersion: monitoring.gdc.goog/v1
   kind: MonitoringTarget
   # [...]
   status:
     conditions:
     - lastTransitionTime: "2023-10-27T15:27:51Z"
       message: ""
       observedGeneration: 1
       reason: Ready
       status: True
       type: Ready
   ```
4. [Query and view metrics on dashboards](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/gdcag/application/ao-user/query-and-view-metrics#query-and-view-metrics-on-dashboards)
   to confirm that your metrics server is sending the expected data.

   > [!NOTE]
   > **Note:** If metrics are not showing up, resolve any issues with your `MonitoringTarget` configuration before enabling mTLS. Review the [Collect metrics](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/gdcag/application/ao-user/collect-metrics) documentation to ensure your configuration is correct.

## Mount the certificate

Your workload needs to use a certificate that is automatically generated when
you enable mTLS on the `MonitoringTarget` custom resource. Add a volume and
volume mount for this certificate using the following template:

```yaml
apiVersion: apps/v1
kind: Deployment # or StatefulSet
metadata:
  name: "SERVICE_NAME"
spec:
  template:
    spec:
      containers:
        - name: "CONTAINER_NAME"
          volumeMounts:
          - mountPath: "/etc/ssl/MONITORING_TARGET_NAME-cert"
            name: "MONITORING_TARGET_NAME-cert"
            readOnly: true
      volumes:
      - name: "MONITORING_TARGET_NAME-cert"
        secret:
          secretName: "MONITORING_TARGET_NAME-cert"
  # [...]
```

Replace the following:

- `SERVICE_NAME`: the name of your `Deployment` or `StatefulSet` object.
- `CONTAINER_NAME`: the name of the container where you are adding the volume mount.
- `MONITORING_TARGET_NAME`: the name of your `MonitoringTarget` custom resource. The secret name is based on this name. For example, a `my-mon-target` resource creates a `my-mon-target-cert` secret name.

## Serve metrics

Your metrics server must serve metrics from an mTLS-enabled HTTP server. You
might want to encrypt metrics in one of the following two scenarios:

- **For applications you own**: Modify the source code to support mTLS and configure the metrics server with the mTLS certificate.
- **For applications you don't own** : Consult the application's documentation to enable an HTTPS metrics endpoint. Apply the necessary configuration, like configuring command-line arguments or deploying a `config` file.

## Enable mTLS metrics collection

Add the following label to the `metadata` section of your `MonitoringTarget`
custom resource to enable the collection of encrypted metrics:

```yaml
monitoring.gdc.goog/enable-mtls: "true"
```

The custom resource must look like the following example:

```yaml
  apiVersion: monitoring.gdc.goog/v1
  kind: MonitoringTarget
  metadata:
    namespace: my-project-namespace
    name: "SERVICE_NAME"
    labels:
      monitoring.gdc.goog/enable-mtls: "true" # Enable mTLS metrics collection
  spec:
    # [...]
```

To verify that you enabled mTLS authentication, check that your
`MonitoringTarget` custom resource shows a `Ready` status for the certificate
and the certificate secret, like in the following example:

```yaml
apiVersion: monitoring.gdc.goog/v1
kind: MonitoringTarget
# [...]
status:
  conditions:
  - lastTransitionTime: "2023-11-09T11:15:10Z"
    message: "admin,user-vm-1,user-vm-2,org-1-system"
    observedGeneration: 2
    reason: Ready
    status: "True"
    type: Ready
  - lastTransitionTime: "2023-11-09T11:14:43Z"
    message: "Certificate is ready"
    observedGeneration: 2
    reason: Ready
    status: "True"
    type: CertificateReady
  - lastTransitionTime: "2023-11-09T11:15:10Z"
    message: "Successfully created secret"
    observedGeneration: 2
    reason: Ready
    status: "True"
    type: CertificateSecretReady
```

> [!NOTE]
> **Note:** Certificates reside in the org infrastructure cluster. Only certificate *secrets* exist in Kubernetes clusters.