Logging and Audit Logging (LOG and AL) audit logs in
Google Distributed Cloud (GDC) air-gapped appliance let you track administrative and data access
operations for logging and monitoring pipelines in root and organization
workloads. Kubernetes audit logs and the observability audit logger record the
user or service identity, target resource, and JSON payload fields for
`LoggingTarget`, `AuditLoggingTarget`, `LoggingRule`, and
`ObservabilityPipeline` custom resources and for Loki and Prometheus queries.

|---|---|
| **Audit log source** | - [Observability audit logger](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/platform-application/pa-ao-operations/audit-observability-logger) - [Kubernetes audit logs](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/platform-application/pa-ao-operations/audit-kubernetes) <br /> |
| **Audited operations** | - [Run LogQL queries or export logs using the user interface of the monitoring instance](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/platform-application/pa-ao-operations/audit-log#logql-monitoring-instance) - [Perform actions on the `LoggingTarget` custom resource](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/platform-application/pa-ao-operations/audit-log#loggingtarget) - [Perform actions on the `AuditLoggingTarget` custom resource](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/platform-application/pa-ao-operations/audit-log#auditloggingtarget) - [Perform actions on the `LoggingRule` custom resource](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/platform-application/pa-ao-operations/audit-log#loggingrule) - [Perform actions on the `ObservabilityPipeline` custom resource](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/platform-application/pa-ao-operations/audit-log#observability) |

## Run LogQL queries or export logs using the user interface of the monitoring instance

> [!NOTE]
> **Note:** You can export all queries in the monitoring instance. That action doesn't separately capture the export from the user interface, but it logs the executed query.

|---|---|
| **Audit log source** | Proxy server |
| **Log type** | Data plane |

| Fields in the log entry that contain audit information |||
| Audit metadata | Audit field name | Value |
|---|---|---|
| **User or service identity** | `user` | For example, <br /> ```json "user": { "issuer": "https://ais-core.org-1.zone1.google.gdch.test", "identity": "fop-infrastructure-operator@example.com" } ``` <br /> |
| **Target** *(Fields and values that call the API)* | `resource` | For example, `"resource": "/infra-obs/grafana/api/ds/query"` |
| **Action** *(Fields containing the performed operation)* | `action` | Possible values: - `"action": "QUERY"` - `"action": "CREATE"` - `"action": "READ"` - `"action": "UPDATE"` - `"action": "DELETE"` - `"action": "CREATE/UPDATE"` <br /> |
| **Event timestamp** | `time` | For example, `"time": "2022-12-02T21:37:03.657277582Z"` |
| **Source of action** | - `sourceIPs` - `_gdch_service_name` | For example, <br /> ```json "sourceIPs": [ [ "10.253.165.26", "127.0.0.6" ], "_gdch_service_name": "grafana" ``` <br /> |
| **Outcome** | `response` | For example, `"response": "Successful: 200 OK"` |
| **Other fields** | `description` | The `description` value contains the complete query. For more information. see the **Example log**. |

**Example log**

    {
      "sourceIPs": [
        "10.253.165.26",
        "127.0.0.6"
      ],
      "description": "{
        \"queries\":
          [{
            \"refId\":\"A\",
            \"datasource\":
              {
                \"uid\":\"P762A5DD6F13C8B7A\",
                \"type\":\"loki\"
              },
            \"editorMode\":\"builder\",
            \"expr\":\"{service_name=\\\"grafana\\\"} |= ``\",
            \"queryType\":\"range\",
            \"key\":\"Q-fd978c0c-86fd-4c70-bb38-07737a3be3ad-0\",
            \"maxLines\":1000,
            \"legendFormat\":\"\",
            \"datasourceId\":3,
            \"intervalMs\":500,
            \"maxDataPoints\":1688
          }],
        \"range\":
          {
            \"from\":\"2022-12-02T21:22:03.496Z\",
            \"to\":\"2022-12-02T21:37:03.496Z\",
            \"raw\":{\"from\":\"now-15m\",\"to\":\"now\"}
          },
        \"from\":\"1670016123496\",
        \"to\":\"1670017023496\"
      }",
      "response": "Successful: 200 OK",
      "_gdch_namespace": "infra-obs-obs-system",
      "numBytesSent": 190079,
      "time": "2022-12-02T21:37:03.657277582Z",
      "user": {
        "issuer": "https://ais-core.org-1.zone1.google.gdch.test",
        "identity": "fop-infrastructure-operator@example.com"
      },
      "_gdch_service_name": "grafana",
      "_gdch_service_tenant": "infra-obs",
      "numBytesReceived": 3172,
      "resource": "/infra-obs/grafana/api/ds/query",
      "auditID": "b519ec65-d906-4a79-bcfe-a4e1984045fe",
      "action": "QUERY",
      "_gdch_cluster": "org-1-admin",
      "_gdch_fluentbit_pod": "anthos-audit-logs-forwarder-9x9pd"
    }

## Perform actions on the `LoggingTarget` custom resource

|---|---|
| **Audit log source** | [Kubernetes audit logs](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/platform-application/pa-ao-operations/audit-kubernetes) |
| **Log type** | Control plane |

| Fields in the log entry that contain audit information |||
| Audit metadata | Audit field name | Value |
|---|---|---|
| **User or service identity** | `user` | For example, <br /> ```json "user": { "extra": { "authentication.kubernetes.io/pod-name": [ "fleet-admin-controller-875778d98-99l6n" ], "authentication.kubernetes.io/pod-uid": [ "4800e06c-c96d-4e17-ae1a-b5a74eedf6ee" ] }, "uid": "b18e586e-db0e-417e-9dff-1a722ab36bf4", "username": "system:serviceaccount:gpc-system:fleet-admin-controller", "groups": [ "system:serviceaccounts", "system:serviceaccounts:gpc-system", "system:authenticated" ] } ``` <br /> |
| **Target** *(Fields and values that call the API)* | - `requestURI` - `objectRef` | For example, <br /> ```json "requestURI": "/apis/logging.gdc.goog/v1/namespaces/obs-system/loggingtargets/lt-cfg1", "objectRef": { "uid": "2e540720-ed23-4665-8c40-c399cb6be624", "namespace": "obs-system", "name": "lt-cfg1", "resource": "loggingtargets", "apiVersion": "v1", "apiGroup": "logging.gdc.goog", "resourceVersion": "5326570" } ``` <br /> |
| **Action** *(Fields containing the performed operation)* | `verb` | Possible values: - `"verb": "create"` - `"verb": "delete"` - `"verb": "get"` - `"verb": "list"` - `"verb": "patch"` - `"verb": "update"` - `"verb": "watch"` <br /> |
| **Event timestamp** | `requestReceivedTimestamp` | For example, `"requestReceivedTimestamp": "2022-12-06T14:37:41.035715Z"` |
| **Source of action** | - `sourceIPs` - `_gdch_service_name` | For example, <br /> ```json "sourceIPs": [ "10.253.164.209" ], "_gdch_service_name": "apiserver" ``` <br /> |
| **Outcome** | `responseStatus` | For example, <br /> ```json "responseStatus": { "metadata": {}, "code": 200 } ``` <br /> |
| **Other fields** | *Not applicable* | *Not applicable* |

**Example log**

    {
      "level": "Metadata",
      "auditID": "94c2106f-1fd1-428b-adbc-80ac48ef479e",
      "_gdch_cluster": "org-1-admin",
      "requestURI": "/apis/logging.gdc.goog/v1/namespaces/obs-system/loggingtargets/lt-cfg1",
      "_gdch_fluentbit_pod": "anthos-audit-logs-forwarder-4gwpn",
      "verb": "update",
      "userAgent": "fleet-admin-cm/v0.0.0 (linux/amd64) kubernetes/$Format",
      "responseStatus": {
        "metadata": {},
        "code": 200
      },
      "user": {
        "extra": {
          "authentication.kubernetes.io/pod-name": [
            "fleet-admin-controller-875778d98-99l6n"
          ],
          "authentication.kubernetes.io/pod-uid": [
            "4800e06c-c96d-4e17-ae1a-b5a74eedf6ee"
          ]
        },
        "uid": "b18e586e-db0e-417e-9dff-1a722ab36bf4",
        "username": "system:serviceaccount:gpc-system:fleet-admin-controller",
        "groups": [
          "system:serviceaccounts",
          "system:serviceaccounts:gpc-system",
          "system:authenticated"
        ]
      },
      "annotations": {
        "authorization.k8s.io/decision": "allow",
        "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"fleet-admin-common-controller\" of ClusterRole \"fleet-admin-common-controllers-role\" to ServiceAccount \"fleet-admin-controller/gpc-system\""
      },
      "sourceIPs": [
        "10.253.164.209"
      ],
      "stage": "ResponseComplete",
      "kind": "Event",
      "apiVersion": "audit.k8s.io/v1",
      "stageTimestamp": "2022-12-06T14:37:41.035715Z",
      "objectRef": {
        "uid": "2e540720-ed23-4665-8c40-c399cb6be624",
        "namespace": "obs-system",
        "name": "lt-cfg1",
        "resource": "loggingtargets",
        "apiVersion": "v1",
        "apiGroup": "logging.gdc.goog",
        "resourceVersion": "5326570"
      },
      "requestReceivedTimestamp": "2022-12-06T14:37:40.942762Z",
      "_gdch_service_name": "apiserver"
    }

## Perform actions on the `AuditLoggingTarget` custom resource

|---|---|
| **Audit log source** | [Kubernetes audit logs](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/platform-application/pa-ao-operations/audit-kubernetes) |
| **Log type** | Control plane |

| Fields in the log entry that contain audit information |||
| Audit metadata | Audit field name | Value |
|---|---|---|
| **User or service identity** | `user` | For example, <br /> ```json "user": { "groups": [ "system:masters", "system:authenticated" ], "username": "kubernetes-admin" } ``` <br /> |
| **Target** *(Fields and values that call the API)* | - `requestURI` - `objectRef` | For example, <br /> ```json "requestURI": "/apis/logging.private.gdc.goog/v1/namespaces/obs-system/auditloggingtargets/audit-logging-config-qwerty", "objectRef": { "name": "audit-logging-config-qwerty", "namespace": "obs-system", "resource": "auditloggingtargets", "apiGroup": "logging.private.gdc.goog", "apiVersion": "v1" } ``` <br /> |
| **Action** *(Fields containing the performed operation)* | `verb` | Possible values: - `"verb": "create"` - `"verb": "delete"` - `"verb": "get"` - `"verb": "list"` - `"verb": "patch"` - `"verb": "update"` - `"verb": "watch"` <br /> |
| **Event timestamp** | `requestReceivedTimestamp` | For example, `"requestReceivedTimestamp": "2022-12-05T19:09:56.861278Z"` |
| **Source of action** | - `sourceIPs` - `_gdch_service_name` | For example, <br /> ```json "sourceIPs": [ "10.200.0.7" ], "_gdch_service_name": "apiserver" ``` <br /> |
| **Outcome** | `responseStatus` | For example, <br /> ```json "responseStatus": { "metadata": {}, "code": 200 } ``` <br /> |
| **Other fields** | *Not applicable* | *Not applicable* |

**Example log**

    {
        "level": "Metadata",
        "auditID": "1fd8d531-c488-4478-8341-47346b1b6eda",
        "stageTimestamp": "2022-12-05T19:09:56.861278Z",
        "stage": "RequestReceived",
        "_gdch_cluster": "org-1-admin",
        "verb": "delete",
        "_gdch_fluentbit_pod": "anthos-audit-logs-forwarder-56tbb",
        "requestReceivedTimestamp": "2022-12-05T19:09:56.861278Z",
        "objectRef": {
            "name": "audit-logging-config-qwerty",
            "namespace": "obs-system",
            "resource": "auditloggingtargets",
            "apiGroup": "logging.private.gdc.goog",
            "apiVersion": "v1"
        },
        "user": {
            "groups": [
                "system:masters",
                "system:authenticated"
            ],
            "username": "kubernetes-admin"
        },
        "sourceIPs": [
            "10.200.0.7"
        ],
        "kind": "Event",
        "userAgent": "k9s/v0.0.0 (linux/amd64) kubernetes/$Format",
        "apiVersion": "audit.k8s.io/v1",
        "requestURI": "/apis/logging.private.gdc.goog/v1/namespaces/obs-system/auditloggingtargets/audit-logging-config-qwerty",
        "_gdch_service_name": "apiserver"
    }

## Perform actions on the `LoggingRule` custom resource

|---|---|
| **Audit log source** | [Kubernetes audit logs](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/platform-application/pa-ao-operations/audit-kubernetes) |
| **Log type** | Control plane |

| Fields in the log entry that contain audit information |||
| Audit metadata | Audit field name | Value |
|---|---|---|
| **User or service identity** | `user` | For example, <br /> ```json "user": { "extra": { "authentication.kubernetes.io/pod-name": [ "fleet-admin-controller-875778d98-99l6n" ], "authentication.kubernetes.io/pod-uid": [ "4800e06c-c96d-4e17-ae1a-b5a74eedf6ee" ] }, "uid": "b18e586e-db0e-417e-9dff-1a722ab36bf4", "username": "system:serviceaccount:gpc-system:fleet-admin-controller", "groups": [ "system:serviceaccounts", "system:serviceaccounts:gpc-system", "system:authenticated" ] } ``` <br /> |
| **Target** *(Fields and values that call the API)* | - `requestURI` - `objectRef` | For example, <br /> ```json "requestURI": "/apis/logging.gdc.goog/v1/namespaces/obs-system/loggingrules/lr-cfg1", "objectRef": { "uid": "2e540720-ed23-4665-8c40-c399cb6be624", "namespace": "obs-system", "name": "lr-cfg1", "resource": "loggingrules", "apiVersion": "v1", "apiGroup": "logging.gdc.goog", "resourceVersion": "5326570" } ``` <br /> |
| **Action** *(Fields containing the performed operation)* | `verb` | Possible values: - `"verb": "create"` - `"verb": "delete"` - `"verb": "get"` - `"verb": "list"` - `"verb": "patch"` - `"verb": "update"` - `"verb": "watch"` <br /> |
| **Event timestamp** | `requestReceivedTimestamp` | For example, `"requestReceivedTimestamp": "2022-12-06T14:37:41.035715Z"` |
| **Source of action** | - `sourceIPs` - `_gdch_service_name` | For example, <br /> ```json "sourceIPs": [ "10.253.164.209" ], "_gdch_service_name": "apiserver" ``` <br /> |
| **Outcome** | `responseStatus` | For example, <br /> ```json "responseStatus": { "metadata": {}, "code": 200 } ``` <br /> |
| **Other fields** | *Not applicable* | *Not applicable* |

**Example log**

    {
      "level": "Metadata",
      "auditID": "94c2106f-1fd1-428b-adbc-80ac48ef479e",
      "_gdch_cluster": "org-1-admin",
      "requestURI": "/apis/logging.gdc.goog/v1/namespaces/obs-system/loggingrules/lr-cfg1",
      "_gdch_fluentbit_pod": "anthos-audit-logs-forwarder-4gwpn",
      "verb": "update",
      "userAgent": "fleet-admin-cm/v0.0.0 (linux/amd64) kubernetes/$Format",
      "responseStatus": {
        "metadata": {},
        "code": 200
      },
      "user": {
        "extra": {
          "authentication.kubernetes.io/pod-name": [
            "fleet-admin-controller-875778d98-99l6n"
          ],
          "authentication.kubernetes.io/pod-uid": [
            "4800e06c-c96d-4e17-ae1a-b5a74eedf6ee"
          ]
        },
        "uid": "b18e586e-db0e-417e-9dff-1a722ab36bf4",
        "username": "system:serviceaccount:gpc-system:fleet-admin-controller",
        "groups": [
          "system:serviceaccounts",
          "system:serviceaccounts:gpc-system",
          "system:authenticated"
        ]
      },
      "annotations": {
        "authorization.k8s.io/decision": "allow",
        "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"fleet-admin-common-controller\" of ClusterRole \"fleet-admin-common-controllers-role\" to ServiceAccount \"fleet-admin-controller/gpc-system\""
      },
      "sourceIPs": [
        "10.253.164.209"
      ],
      "stage": "ResponseComplete",
      "kind": "Event",
      "apiVersion": "audit.k8s.io/v1",
      "stageTimestamp": "2022-12-06T14:37:41.035715Z",
      "objectRef": {
        "uid": "2e540720-ed23-4665-8c40-c399cb6be624",
        "namespace": "obs-system",
        "name": "lr-cfg1",
        "resource": "loggingrules",
        "apiVersion": "v1",
        "apiGroup": "logging.gdc.goog",
        "resourceVersion": "5326570"
      },
      "requestReceivedTimestamp": "2022-12-06T14:37:40.942762Z",
      "_gdch_service_name": "apiserver"
    }

## Perform actions on the `ObservabilityPipeline` custom resource

|---|---|
| **Audit log source** | [Kubernetes audit logs](https://docs.cloud.google.com/distributed-cloud/hosted/docs/latest/appliance/platform-application/pa-ao-operations/audit-kubernetes) |
| **Log type** | Control plane |

| Fields in the log entry that contain audit information |||
| Audit metadata | Audit field name | Value |
|---|---|---|
| **User or service identity** | `user` | For example, <br /> ```json "user": { "username": "kubernetes-admin", "groups": [ "system:masters", "system:authenticated" ] } ``` <br /> |
| **Target** *(Fields and values that call the API)* | - `requestURI` - `objectRef` | For example, <br /> ```json "requestURI": "/apis/observability.gdc.goog/v1/namespaces/obs-system/observabilitypipelines/default", "objectRef": { "apiGroup": "observability.gdc.goog", "apiVersion": "v1", "name": "default", "resource": "observabilitypipelines", "namespace": "obs-system" } ``` <br /> |
| **Action** *(Fields containing the performed operation)* | `verb` | Possible values: - `"verb": "create"` - `"verb": "delete"` - `"verb": "get"` - `"verb": "list"` - `"verb": "patch"` - `"verb": "update"` - `"verb": "watch"` <br /> |
| **Event timestamp** | `requestReceivedTimestamp` | For example, `"requestReceivedTimestamp": "2022-12-06T14:58:03.742024Z"` |
| **Source of action** | - `sourceIPs` - `_gdch_service_name` | For example, <br /> ```json "sourceIPs": [ "10.200.0.7" ], "_gdch_service_name": "apiserver" ``` <br /> |
| **Outcome** | `responseStatus` | For example, <br /> ```json "responseStatus": { "metadata": {}, "code": 200 } ``` <br /> |
| **Other fields** | *Not applicable* | *Not applicable* |

**Example log**

    {
        "_gdch_fluentbit_pod": "anthos-audit-logs-forwarder-sff9t",
        "requestURI": "/apis/observability.gdc.goog/v1/namespaces/obs-system/observabilitypipelines/default",
        "stage": "RequestReceived",
        "verb": "get",
        "user": {
            "username": "kubernetes-admin",
            "groups": [
                "system:masters",
                "system:authenticated"
            ]
        },
        "objectRef": {
            "apiGroup": "observability.gdc.goog",
            "apiVersion": "v1",
            "name": "default",
            "resource": "observabilitypipelines",
            "namespace": "obs-system"
        },
        "requestReceivedTimestamp": "2022-12-06T14:58:03.742024Z",
        "stageTimestamp": "2022-12-06T14:58:03.742024Z",
        "level": "Metadata",
        "auditID": "f5a473cf-65c9-4706-aa0e-e657c9f308f7",
        "_gdch_cluster": "org-1-admin",
        "userAgent": "kubectl/v1.25.4 (linux/amd64) kubernetes/872a965",
        "sourceIPs": [
            "10.200.0.7"
        ],
        "kind": "Event",
        "apiVersion": "audit.k8s.io/v1",
        "_gdch_service_name": "apiserver"
    }