This page shows you how to troubleshoot and verify Google Kubernetes Engine (GKE)
ambient networking. It provides instructions for checking the health of
in-cluster ambient components, diagnosing traffic authentication and
authorization issues, verifying certificate issuance, and inspecting metrics and
logs.

## Verify components are installed and healthy

In-cluster components for Cloud Service Mesh ambient mode are installed in
the `gke-managed-ambient` namespace. You can see the status of all relevant Pods
with the following command:

```bash
kubectl get pods -n gke-managed-ambient
```

## Verify traffic is authenticated and authorized

Using [Logs Explorer](https://docs.cloud.google.com/logging/docs/view/logs-explorer-interface.md.txt) in
Google Cloud console, check the access logs that correspond to the connections of
interest:

    resource.type="k8s_container"
    resource.labels.location="CLUSTER_LOCATION"
    resource.labels.cluster_name="CLUSTER_NAME"
    logName="projects/PROJECT_ID/logs/gke-ambient-node-proxy-accesslog"

The access logs display connection details, including authentication and
authorization status.

### Log format

The following fields are present in the `jsonPayload` of the access log entry:

| Field | Type | Description |
|---|---|---|
| `connection` | struct | Connection information. |
| `connection.src_ip` | string | Source IP of the connection. |
| `connection.src_port` | int | Source port of the connection. |
| `connection.dest_ip` | string | Destination IP of the connection. |
| `connection.dest_port` | int | Destination port of the connection. |
| `connection.direction` | string | Direction of the connection: `{ingress, egress}` |
| `src` | struct | Endpoint information of the source. |
| `src.spiffe_id` | string | The SPIFFE identity of the Pod. Empty if the Pod does not have a SPIFFE identity. |
| `dst` | struct | Endpoint information of the destination. |
| `dst.spiffe_id` | string | The SPIFFE identity of the Pod. Empty if the Pod does not have a SPIFFE identity. |
| `error_details` | string | Details about the error that occurred (or `-` if no error occurred). |
| `timestamp` | string | Time when the connection was observed (UTC). |
| `tls_failure_reason` | string | TLS handshake [failure reason](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/security/ssl#trouble-shooting). `-` if the connection is not TLS. |

## Verify certificates are successfully issued

Use the following command to inspect the status of `CertificateSigningRequests`
for workload Pods in your cluster:

```bash
kubectl get csr
```

You should see "Approved,Issued" conditions similar to the following:

    NAME                           AGE   SIGNERNAME                 CONDITION
    gke-ambient-spiffe-csr-q8kqh   2s    spiffe.gke.io/fleet-svid   Approved,Issued
    gke-ambient-spiffe-csr-t5px2   7s    spiffe.gke.io/fleet-svid   Approved,Issued

## Unable to view Layer 4 metrics

In addition to standard Cloud Observability metrics, Cloud Service Mesh
ambient networking integrates with Network Services Monitoring to provide
detailed Layer 4 network telemetry.

If services or Layer 4 metrics don't appear in Network Services Monitoring or
Cloud Monitoring, verify that both `networking.gke.io/dataplane-mode=ambient`
and `networking.gke.io/ambient-network-metrics=enabled` are set on the workload namespace and that active traffic is flowing between enrolled Pods.

For more information, see the
[Network Services Monitoring overview](https://docs.cloud.google.com/network-intelligence-center/docs/network-services-monitoring/overview.md.txt) and the
[Network Services Monitoring metrics reference](https://docs.cloud.google.com/network-intelligence-center/docs/network-services-monitoring/metrics-reference.md.txt).