Application Monitoring supported infrastructure

The following table lists the infrastructure resources that are integrated with Application Monitoring. When these resources are registered as an App Hub service or workload, the telemetry that the resources generate includes application-specific labels. This telemetry includes platform and audit log entries, metric data, and trace data. For a list of services and workloads that are integrated with App Hub, see App Hub supported resources.

The dashboards that Application Monitoring creates display log and metric data, including the following golden signals, when that data includes application-specific labels:

• Traffic: Incoming request rates on the service or workload over the selected time period. For some configurations, this chart contains tabs. The Request tab displays the networking L7 metric and the Bytes tab displays the networking L4 metric.
• Server error rate: Average percentage of incoming requests that generate or map to 5xx HTTP response codes over the selected time period.
• P95 latency: 95th percentile of latency for a request served over the selected time period, in milliseconds.
• Saturation: Measures how full your service or workload is. For example, for managed instance groups (MIGs), Cloud Run, and Google Kubernetes Engine deployments, this field shows the CPU utilization.

If you explore a dashboard, you might notice that a charted metric for a golden signal differs from the metric you expect. For some golden signals, Google Cloud Observability has a prioritized list of source metrics. If a preferred metric isn't available, then Google Cloud Observability searches the list until it finds an available metric.

Supported infrastructure resources

The Notes and limitations column lists details about which golden signals are supported. This column also lists limitations.

Infrastructure resources	Golden signals	Notes and limitations
AlloyDB for PostgreSQL clusters (Regional)		Application labels aren't attached to spans.
AlloyDB for PostgreSQL instances (Regional)	Traffic: instance/postgres/transaction_count Server error rate: Ratio of the instance/postgres/abort_count to the total instance/postgres/transaction_count. Saturation: instance/cpu/average_utilization	Application labels aren't attached to spans.
Bigtable clusters (Regional)	Saturation: cluster/cpu_load	Application labels aren't attached to spans.
Bigtable instances (Regional)		Application labels aren't attached to metric data or to spans.
Cloud Domains		Application labels aren't attached to metric data or to spans.
Cloud Run jobs (Regional)	Saturation: container/cpu/utilizations
Cloud Run services and functions (Regional)	Traffic: request_count Server error rate: The ratio of the number of requests with a response status of 5xx to the total number of requests. P95 latency: request_latencies Saturation: container/cpu/utilizations	Only trace spans generated by instrumented workloads contain application labels. Only customer-instrumented workloads running within Cloud Run generate golden signals.
Cloud Logging log buckets (Global and regional)		Only audit logs contain application labels.
Cloud SQL instances (Regional)	Saturation: database/cpu/utilization	Application labels aren't attached to spans.
Cloud Storage buckets (Regional and multi-regional)	Traffic: api/request_count Server error rate: Ratio of requests with a response status that indicates an error to the total number of requests. The following status values indicate an error: DATA_LOSS DEADLINE_EXCEEDED INTERNAL UNAVAILABLE UNIMPLEMENTED UNKNOWN	Application labels aren't attached to spans.
Compute Engine MIGs (Regional and zonal)	Traffic: instance/network/received_bytes_count Saturation: instance/cpu/utilization
Dataproc Metastore service (Regional and multi-regional)	Traffic: The metric depends on whether the service is single- or multi-region: Single: service/request_count. Multi: service/multi_region/request_count. Server error rate: The ratio of the number of service/grpc/finished_requests_count with a response status that indicates an error to the total number of finished requests. The following status values indicate an error: DEADLINE_EXCEEDED INTERNAL UNAVAILABLE UNIMPLEMENTED P95 latency: The metric depends on whether the service is single- or multi-region: Single: service/request_latencies Multi: service/multi_region/request_latencies.	Application labels aren't attached to spans. For general information, see Dataproc Metastore overview.
Cloud Deploy delivery pipelines (Regional)		Application labels aren't attached to spans.
Firestore databases (Regional)	Traffic: api/request_count Server error rate: Ratio of requests with a response status that indicates an error to the total number of requests. The following status values indicate an error: DEADLINE_EXCEEDED INTERNAL UNAVAILABLE UNIMPLEMENTED P95 latency: api/request_latencies	Application labels aren't attached to spans.
GKE Gateway gateway.networking.k8s.io/Gateway	Traffic: application_lb/request_count Server error rate: The ratio of the number of requests with a response status of 5xx to the total number of requests. P95 latency: application_lb/total_latencies	Application labels aren't attached to log entries or to spans.
GKE Ingress networking.k8s.io/Ingress	Traffic: application_lb/request_count Server error rate: The ratio of the number of requests with a response status of 5xx to the total number of requests. P95 latency: application_lb/total_latencies	Application labels aren't attached to spans.
Kubernetes CronJobs (Regional and zonal)		To learn how to run Cron Jobs in GKE, see About CronJobs in GKE.
Kubernetes DaemonSets (Regional and zonal)	See Golden signals for Kubernetes.
Kubernetes Deployments (Regional and zonal)	See Golden signals for Kubernetes.
Kubernetes StatefulSets (Regional and zonal)	See Golden signals for Kubernetes.	To learn how to use a StatefulSet or deploy a stateful application, see About StatefulSets in Google Kubernetes Engine.
Layer 7 external and internal Application Load Balancers with HTTP or HTTPS traffic (Global, regional, and cross region)	Traffic: Based on a Cloud Load Balancing metric type that records the request count, like https/request_count. The configuration of the Cloud Load Balancing determines the actual metric. Server error rate: The ratio of the number of requests with a response status of 5xx to the total number of requests. P95 latency: Based on a Cloud Load Balancing metric type that records total latencies, like https/total_latencies. The configuration of the Cloud Load Balancing determines the actual metric.	For Application Load Balancers, only forwarding rules and backend services are integrated with App Hub. Application labels aren't attached to spans.
Memorystore for Redis clusters (Regional)	Saturation: cluster/cpu/average_utilization	Application labels aren't attached to log entries or to spans.
Memorystore for Redis instances (Regional)		Application labels aren't attached to metric data or to spans.
Pub/Sub topics (Global)	Traffic: topic/send_request_count Server error rate: The ratio of the number of requests with a response code of internal to the total number of requests. P95 latency: topic/send_request_latencies	Application labels aren't attached to spans.
Pub/Sub subscription (Global)	Traffic: subscription/pull_request_count Server error rate: The ratio of the number of requests with a response code of internal to the total number of requests. P95 latency: subscription/push_request_latencies	Application labels aren't attached to spans.
Secret Manager secrets (Global and regional)		Application labels aren't attached to spans.
Spanner databases (Global and regional)		Application labels aren't attached to metric data or to spans.
Spanner instances (Global and regional)	Traffic: api/api_request_count Server error rate: Ratio of requests with a response status that indicates an error to the total number of requests. The following status values indicate an error: data_loss deadline_exceeded internal unavailable unimplemented unknown P95 latency: api/request_latencies Saturation: instance/cpu/utilization	Application labels aren't attached to spans.
Vertex AI Agent Engine agents (Regional)	Traffic: reasoning_engine/request_count Server error rate: Ratio of requests whose response status is at least 500 and no more than 599 to the total number of requests. P95 latency: reasoning_engine/request_latencies

Golden signals for Kubernetes

For Kubernetes DaemonSets, Deployments, and StatefulSets, Google Cloud Observability uses the following metric data to derive values for golden signals, when that metric data is available:

• Traffic: service/server/request_count. The units for this metric are requests per second.
• Server error rate: Ratio of service/server/request_count requests whose response status is at least 500 and no more than 599 to the total number of requests.
• P95 latency: The 95th percentile from the service/server/response_latencies.
• Saturation: The ratio of the container/cpu/core_usage_time to the container/cpu/request_cores.

On a dashboard, you might see that the chart that displays traffic information contains two tabs:

• The tab labeled Traffic (Requests) displays networking L7 traffic data and the units are requests per second.
• The tab labeled Traffic (Bytes) displays the pod/network/received_bytes_count metric, which has units KiB per second and is a networking L4 metric.

When both the L7 and L4 traffic metrics are available, the L7 metrics appear on the summary pages but the traffic chart contains tabs, which let you view both metrics. If only one metric is available, then that metric is shown on the summary page and on dashboards.

For workloads that run on GKE, Google Cloud Observability might derive golden signals from the Prometheus metric http_server_request_duration_seconds, which is only available when you instrument your application by using OpenTelemetry. To learn more, see Instrument your application.