Logging and Audit Logging (LOG and AL) audit logs in
Google Distributed Cloud (GDC) air-gapped appliance let you track administrative and data access
operations for logging and monitoring pipelines in root and organization
workloads. Kubernetes audit logs and the observability audit logger record the
user or service identity, target resource, and JSON payload fields for
LoggingTarget, AuditLoggingTarget, LoggingRule, and
ObservabilityPipeline custom resources and for Loki and Prometheus queries.
Run LogQL queries or export logs using the user interface of the monitoring instance
| Audit log source |
Proxy server |
| Log type |
Data plane |
| Fields in the log entry that contain audit information | ||
|---|---|---|
| Audit metadata | Audit field name | Value |
| User or service identity | user |
For example, "user": { "issuer": "https://ais-core.org-1.zone1.google.gdch.test", "identity": "fop-infrastructure-operator@example.com" } |
|
Target (Fields and values that call the API) |
resource |
For example,
|
|
Action (Fields containing the performed operation) |
action |
Possible values:
|
| Event timestamp |
time
|
For example,
|
| Source of action |
|
For example, "sourceIPs": [ [ "10.253.165.26", "127.0.0.6" ], "_gdch_service_name": "grafana" |
| Outcome | response |
For example,
|
| Other fields | description |
The description value contains the complete query. For more information. see the Example log. |
Example log
{
"sourceIPs": [
"10.253.165.26",
"127.0.0.6"
],
"description": "{
\"queries\":
[{
\"refId\":\"A\",
\"datasource\":
{
\"uid\":\"P762A5DD6F13C8B7A\",
\"type\":\"loki\"
},
\"editorMode\":\"builder\",
\"expr\":\"{service_name=\\\"grafana\\\"} |= ``\",
\"queryType\":\"range\",
\"key\":\"Q-fd978c0c-86fd-4c70-bb38-07737a3be3ad-0\",
\"maxLines\":1000,
\"legendFormat\":\"\",
\"datasourceId\":3,
\"intervalMs\":500,
\"maxDataPoints\":1688
}],
\"range\":
{
\"from\":\"2022-12-02T21:22:03.496Z\",
\"to\":\"2022-12-02T21:37:03.496Z\",
\"raw\":{\"from\":\"now-15m\",\"to\":\"now\"}
},
\"from\":\"1670016123496\",
\"to\":\"1670017023496\"
}",
"response": "Successful: 200 OK",
"_gdch_namespace": "infra-obs-obs-system",
"numBytesSent": 190079,
"time": "2022-12-02T21:37:03.657277582Z",
"user": {
"issuer": "https://ais-core.org-1.zone1.google.gdch.test",
"identity": "fop-infrastructure-operator@example.com"
},
"_gdch_service_name": "grafana",
"_gdch_service_tenant": "infra-obs",
"numBytesReceived": 3172,
"resource": "/infra-obs/grafana/api/ds/query",
"auditID": "b519ec65-d906-4a79-bcfe-a4e1984045fe",
"action": "QUERY",
"_gdch_cluster": "org-1-admin",
"_gdch_fluentbit_pod": "anthos-audit-logs-forwarder-9x9pd"
}
Perform actions on the LoggingTarget custom resource
| Audit log source | |
| Log type |
Control plane |
| Fields in the log entry that contain audit information | ||
|---|---|---|
| Audit metadata | Audit field name | Value |
| User or service identity | user |
For example, "user": { "extra": { "authentication.kubernetes.io/pod-name": [ "fleet-admin-controller-875778d98-99l6n" ], "authentication.kubernetes.io/pod-uid": [ "4800e06c-c96d-4e17-ae1a-b5a74eedf6ee" ] }, "uid": "b18e586e-db0e-417e-9dff-1a722ab36bf4", "username": "system:serviceaccount:gpc-system:fleet-admin-controller", "groups": [ "system:serviceaccounts", "system:serviceaccounts:gpc-system", "system:authenticated" ] } |
|
Target (Fields and values that call the API) |
|
For example, "requestURI": "/apis/logging.gdc.goog/v1/namespaces/obs-system/loggingtargets/lt-cfg1", "objectRef": { "uid": "2e540720-ed23-4665-8c40-c399cb6be624", "namespace": "obs-system", "name": "lt-cfg1", "resource": "loggingtargets", "apiVersion": "v1", "apiGroup": "logging.gdc.goog", "resourceVersion": "5326570" } |
|
Action (Fields containing the performed operation) |
verb |
Possible values:
|
| Event timestamp |
requestReceivedTimestamp
|
For example,
|
| Source of action |
|
For example, "sourceIPs": [ "10.253.164.209" ], "_gdch_service_name": "apiserver" |
| Outcome | responseStatus |
For example, "responseStatus": { "metadata": {}, "code": 200 } |
| Other fields | Not applicable | Not applicable |
Example log
{
"level": "Metadata",
"auditID": "94c2106f-1fd1-428b-adbc-80ac48ef479e",
"_gdch_cluster": "org-1-admin",
"requestURI": "/apis/logging.gdc.goog/v1/namespaces/obs-system/loggingtargets/lt-cfg1",
"_gdch_fluentbit_pod": "anthos-audit-logs-forwarder-4gwpn",
"verb": "update",
"userAgent": "fleet-admin-cm/v0.0.0 (linux/amd64) kubernetes/$Format",
"responseStatus": {
"metadata": {},
"code": 200
},
"user": {
"extra": {
"authentication.kubernetes.io/pod-name": [
"fleet-admin-controller-875778d98-99l6n"
],
"authentication.kubernetes.io/pod-uid": [
"4800e06c-c96d-4e17-ae1a-b5a74eedf6ee"
]
},
"uid": "b18e586e-db0e-417e-9dff-1a722ab36bf4",
"username": "system:serviceaccount:gpc-system:fleet-admin-controller",
"groups": [
"system:serviceaccounts",
"system:serviceaccounts:gpc-system",
"system:authenticated"
]
},
"annotations": {
"authorization.k8s.io/decision": "allow",
"authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"fleet-admin-common-controller\" of ClusterRole \"fleet-admin-common-controllers-role\" to ServiceAccount \"fleet-admin-controller/gpc-system\""
},
"sourceIPs": [
"10.253.164.209"
],
"stage": "ResponseComplete",
"kind": "Event",
"apiVersion": "audit.k8s.io/v1",
"stageTimestamp": "2022-12-06T14:37:41.035715Z",
"objectRef": {
"uid": "2e540720-ed23-4665-8c40-c399cb6be624",
"namespace": "obs-system",
"name": "lt-cfg1",
"resource": "loggingtargets",
"apiVersion": "v1",
"apiGroup": "logging.gdc.goog",
"resourceVersion": "5326570"
},
"requestReceivedTimestamp": "2022-12-06T14:37:40.942762Z",
"_gdch_service_name": "apiserver"
}
Perform actions on the AuditLoggingTarget custom resource
| Audit log source | |
| Log type |
Control plane |
| Fields in the log entry that contain audit information | ||
|---|---|---|
| Audit metadata | Audit field name | Value |
| User or service identity | user |
For example, "user": { "groups": [ "system:masters", "system:authenticated" ], "username": "kubernetes-admin" } |
|
Target (Fields and values that call the API) |
|
For example, "requestURI": "/apis/logging.private.gdc.goog/v1/namespaces/obs-system/auditloggingtargets/audit-logging-config-qwerty", "objectRef": { "name": "audit-logging-config-qwerty", "namespace": "obs-system", "resource": "auditloggingtargets", "apiGroup": "logging.private.gdc.goog", "apiVersion": "v1" } |
|
Action (Fields containing the performed operation) |
verb |
Possible values:
|
| Event timestamp |
requestReceivedTimestamp
|
For example,
|
| Source of action |
|
For example, "sourceIPs": [ "10.200.0.7" ], "_gdch_service_name": "apiserver" |
| Outcome | responseStatus |
For example, "responseStatus": { "metadata": {}, "code": 200 } |
| Other fields | Not applicable | Not applicable |
Example log
{
"level": "Metadata",
"auditID": "1fd8d531-c488-4478-8341-47346b1b6eda",
"stageTimestamp": "2022-12-05T19:09:56.861278Z",
"stage": "RequestReceived",
"_gdch_cluster": "org-1-admin",
"verb": "delete",
"_gdch_fluentbit_pod": "anthos-audit-logs-forwarder-56tbb",
"requestReceivedTimestamp": "2022-12-05T19:09:56.861278Z",
"objectRef": {
"name": "audit-logging-config-qwerty",
"namespace": "obs-system",
"resource": "auditloggingtargets",
"apiGroup": "logging.private.gdc.goog",
"apiVersion": "v1"
},
"user": {
"groups": [
"system:masters",
"system:authenticated"
],
"username": "kubernetes-admin"
},
"sourceIPs": [
"10.200.0.7"
],
"kind": "Event",
"userAgent": "k9s/v0.0.0 (linux/amd64) kubernetes/$Format",
"apiVersion": "audit.k8s.io/v1",
"requestURI": "/apis/logging.private.gdc.goog/v1/namespaces/obs-system/auditloggingtargets/audit-logging-config-qwerty",
"_gdch_service_name": "apiserver"
}
Perform actions on the LoggingRule custom resource
| Audit log source | |
| Log type |
Control plane |
| Fields in the log entry that contain audit information | ||
|---|---|---|
| Audit metadata | Audit field name | Value |
| User or service identity | user |
For example, "user": { "extra": { "authentication.kubernetes.io/pod-name": [ "fleet-admin-controller-875778d98-99l6n" ], "authentication.kubernetes.io/pod-uid": [ "4800e06c-c96d-4e17-ae1a-b5a74eedf6ee" ] }, "uid": "b18e586e-db0e-417e-9dff-1a722ab36bf4", "username": "system:serviceaccount:gpc-system:fleet-admin-controller", "groups": [ "system:serviceaccounts", "system:serviceaccounts:gpc-system", "system:authenticated" ] } |
|
Target (Fields and values that call the API) |
|
For example, "requestURI": "/apis/logging.gdc.goog/v1/namespaces/obs-system/loggingrules/lr-cfg1", "objectRef": { "uid": "2e540720-ed23-4665-8c40-c399cb6be624", "namespace": "obs-system", "name": "lr-cfg1", "resource": "loggingrules", "apiVersion": "v1", "apiGroup": "logging.gdc.goog", "resourceVersion": "5326570" } |
|
Action (Fields containing the performed operation) |
verb |
Possible values:
|
| Event timestamp |
requestReceivedTimestamp
|
For example,
|
| Source of action |
|
For example, "sourceIPs": [ "10.253.164.209" ], "_gdch_service_name": "apiserver" |
| Outcome | responseStatus |
For example, "responseStatus": { "metadata": {}, "code": 200 } |
| Other fields | Not applicable | Not applicable |
Example log
{
"level": "Metadata",
"auditID": "94c2106f-1fd1-428b-adbc-80ac48ef479e",
"_gdch_cluster": "org-1-admin",
"requestURI": "/apis/logging.gdc.goog/v1/namespaces/obs-system/loggingrules/lr-cfg1",
"_gdch_fluentbit_pod": "anthos-audit-logs-forwarder-4gwpn",
"verb": "update",
"userAgent": "fleet-admin-cm/v0.0.0 (linux/amd64) kubernetes/$Format",
"responseStatus": {
"metadata": {},
"code": 200
},
"user": {
"extra": {
"authentication.kubernetes.io/pod-name": [
"fleet-admin-controller-875778d98-99l6n"
],
"authentication.kubernetes.io/pod-uid": [
"4800e06c-c96d-4e17-ae1a-b5a74eedf6ee"
]
},
"uid": "b18e586e-db0e-417e-9dff-1a722ab36bf4",
"username": "system:serviceaccount:gpc-system:fleet-admin-controller",
"groups": [
"system:serviceaccounts",
"system:serviceaccounts:gpc-system",
"system:authenticated"
]
},
"annotations": {
"authorization.k8s.io/decision": "allow",
"authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"fleet-admin-common-controller\" of ClusterRole \"fleet-admin-common-controllers-role\" to ServiceAccount \"fleet-admin-controller/gpc-system\""
},
"sourceIPs": [
"10.253.164.209"
],
"stage": "ResponseComplete",
"kind": "Event",
"apiVersion": "audit.k8s.io/v1",
"stageTimestamp": "2022-12-06T14:37:41.035715Z",
"objectRef": {
"uid": "2e540720-ed23-4665-8c40-c399cb6be624",
"namespace": "obs-system",
"name": "lr-cfg1",
"resource": "loggingrules",
"apiVersion": "v1",
"apiGroup": "logging.gdc.goog",
"resourceVersion": "5326570"
},
"requestReceivedTimestamp": "2022-12-06T14:37:40.942762Z",
"_gdch_service_name": "apiserver"
}
Perform actions on the ObservabilityPipeline custom resource
| Audit log source | |
| Log type |
Control plane |
| Fields in the log entry that contain audit information | ||
|---|---|---|
| Audit metadata | Audit field name | Value |
| User or service identity | user |
For example, "user": { "username": "kubernetes-admin", "groups": [ "system:masters", "system:authenticated" ] } |
|
Target (Fields and values that call the API) |
|
For example, "requestURI": "/apis/observability.gdc.goog/v1/namespaces/obs-system/observabilitypipelines/default", "objectRef": { "apiGroup": "observability.gdc.goog", "apiVersion": "v1", "name": "default", "resource": "observabilitypipelines", "namespace": "obs-system" } |
|
Action (Fields containing the performed operation) |
verb |
Possible values:
|
| Event timestamp |
requestReceivedTimestamp
|
For example,
|
| Source of action |
|
For example, "sourceIPs": [ "10.200.0.7" ], "_gdch_service_name": "apiserver" |
| Outcome | responseStatus |
For example, "responseStatus": { "metadata": {}, "code": 200 } |
| Other fields | Not applicable | Not applicable |
Example log
{
"_gdch_fluentbit_pod": "anthos-audit-logs-forwarder-sff9t",
"requestURI": "/apis/observability.gdc.goog/v1/namespaces/obs-system/observabilitypipelines/default",
"stage": "RequestReceived",
"verb": "get",
"user": {
"username": "kubernetes-admin",
"groups": [
"system:masters",
"system:authenticated"
]
},
"objectRef": {
"apiGroup": "observability.gdc.goog",
"apiVersion": "v1",
"name": "default",
"resource": "observabilitypipelines",
"namespace": "obs-system"
},
"requestReceivedTimestamp": "2022-12-06T14:58:03.742024Z",
"stageTimestamp": "2022-12-06T14:58:03.742024Z",
"level": "Metadata",
"auditID": "f5a473cf-65c9-4706-aa0e-e657c9f308f7",
"_gdch_cluster": "org-1-admin",
"userAgent": "kubectl/v1.25.4 (linux/amd64) kubernetes/872a965",
"sourceIPs": [
"10.200.0.7"
],
"kind": "Event",
"apiVersion": "audit.k8s.io/v1",
"_gdch_service_name": "apiserver"
}