Logging and monitoring for Cloud Load Balancing callouts

This page shows you how to configure and use Cloud Logging and Cloud Monitoring with Service Extensions callouts for Cloud Load Balancing.

Logging

This section describes logging for Application Load Balancer callouts.

Enable logging on a backend service

You can enable logging for Application Load Balancer callouts while creating the service by enabling logging on the backend service that's the target of a request (and not on the backend service associated with the extension).

To enable logging for the target backend service for a regional internal Application Load Balancer, use the gcloud compute backend-services update command.

gcloud compute backend-services update BACKEND_SERVICE \
    --enable-logging \
    --logging-sample-rate=RATE \
    --region=REGION \
    --logging-optional=LOGGING_OPTIONAL_MODE \
    --logging-optional-fields=OPTIONAL_FIELDS

Replace the following:

BACKEND_SERVICE: the name of the backend service
RATE: a value from 0.0 through 1.0, where 0.0 means no requests are logged and 1.0 means 100% of requests are logged. The default value is 1.0. This setting is effective only when used with the enable-logging parameter. When you omit enable-logging, logging is disabled.
REGION: the region of the backend
LOGGING_OPTIONAL_MODE: enables logging for optional fields in one of these modes:
- INCLUDE_ALL_OPTIONAL includes all optional fields.
- EXCLUDE_ALL_OPTIONAL (default) excludes all optional fields.
- CUSTOM includes a custom list of optional fields.
OPTIONAL_FIELDS: a comma-separated list of optional fields when you select the CUSTOM mode

For more information, see the "Monitor and troubleshoot" pages of the Application Load Balancer documentation, such as Internal Application Load Balancer logging and monitoring.

After you enable logging on the backend service, HTTP or HTTPS requests are logged by using Cloud Logging logs requests.

To view logs, in the Google Cloud console, go to the the Logs Explorer page.

Log messages for a backend service

In general, Application Load Balancer log entries contain information that is useful for monitoring and debugging your HTTP or HTTPS traffic. Log entries contain the following types of information:

Information shown in most Google Cloud logs, such as severity, project ID, project number, and timestamp as described in the LogEntry log.
HttpRequest log fields.

Request logs for HTTP and HTTPS load balancers contain a service_extension_info object in the load balancer log entry JSON payload with the following information:

Field	Type	Description
`backend_target_name`	string	Name of the backend target of the extension.
`backend_target_type`	string	Type of the backend target.
`chain`	string	Name of the extension chain within the service extension resource that matches the request.
`extension`	string	Name of the extension within the extension chain.
`grpc_status`	enum	The most recent status on the gRPC stream. For more information, see gRPC status codes.
`per_processing_request_info`	array	A list of either `ProcessingRequest` stats for `ext_proc` extensions or `CheckRequest` stats for `ext_authz` extensions that occur over the gRPC stream.
`per_processing_request_info[].event_type`	enum	The event type of `ProcessingRequest`. Can be one of these: `REQUEST_HEADERS`, `REQUEST_BODY`, `RESPONSE_HEADERS`, or `RESPONSE_BODY`.
`per_processing_request_info[].latency`	duration	The duration from when the first byte of the `ProcessingRequest` message is sent to the extension to when the last byte of the `ProcessingResponse` message is received.
`resource`	string	Name of the extension resource

Monitoring

This section describes how to monitor callouts configured using Service Extensions for Cloud Load Balancing.

View a Monitoring dashboard

Application Load Balancers export monitoring data to Cloud Monitoring.

Use Monitoring metrics for the following purposes:

Evaluating a load balancer's configuration, usage, and performance
Troubleshooting problems
Improving resource utilization and user experience

To view a predefined dashboard, follow these steps:

In the Google Cloud console, go to the Dashboards overview page.
Go to Dashboards overview
In the Categories section, click GCP.
- To view a list of dashboards for all your load balancers, in the GCP Dashboards list, click the dashboard named Google Cloud Load Balancers. To view a specific load balancer's dashboard, locate the load balancer in the list and click its name.
- To view the predefined dashboards for only your load balancers, select the appropriate dashboard.

In addition to the predefined dashboards in Monitoring, you can create custom dashboards, set up alerts, and query the metrics through the Cloud Monitoring API.

For more information, see the "Monitor and troubleshoot" pages of the Cloud Load Balancing documentation—for example, Internal Application Load Balancer logging and monitoring.

Monitoring metrics for callouts

You can monitor the following metrics for callout backend services.

These metrics have the prefix loadbalancing.googleapis.com/. The prefix is omitted from the entries in the table.

Metric type	Display name Kind, Type, Unit Description
`https/backend_request_count`, `https/external/regional/backend_request_count`, `https/internal/backend_request_count`	Backend request count `DELTA`, `INT64`, `1` The number of times that a callout backend service is called from the Application Load Balancer.
`https/backend_request_bytes_count`, `https/external/regional/backend_request_bytes_count`, `https/internal/backend_request_bytes_count`	Backend request bytes `DELTA`, `INT64`, `By` The number of bytes sent from the load balancer to the callout backend service.
`https/backend_response_bytes_count`, `https/external/regional/backend_response_bytes_count`, `https/internal/backend_response_bytes_count`	Backend response bytes `DELTA`, `INT64`, `By` The number of bytes received by the load balancer from the extension backend.
`https/backend_latencies`, `https/external/regional/backend_latencies`, `https/internal/backend_latencies`	Backend latency `DELTA`, `DISTRIBUTION`, `ms` A distribution calculated from the sum of latencies of each extension invocation between the load balancer and the callout backend service. Sampled every 60 seconds.

What's next

View log messages for callouts