Logging and monitoring for Media CDN plugins

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

This page shows you how to configure and use Cloud Logging and Cloud Monitoring with Service Extensions plugins for Media CDN.

Logging

This section describes Service Extensions logs for Media CDN plugins and how you can view and correlate the information in log messages.

Log messages

Service Extensions supports generating log messages during the execution of your plugin. Recording logs is disabled by default. To record logs for a plugin, enable it when you create the plugin or update it.

Plugin log records are annotated with the following contextual information:

• Standard log annotations, such as timestamp and log level.
• The identity of the plugin that generated the message.
• The plugin callback in which the log message was generated.
• A requestId trace identifier that helps determine the request log that a log message is associated with.

Logs that are pertinent to Service Extensions are in one of the following categories:

• Plugin log messages

  Generated by a logging call, such as info!(...) for Rust, proxywasm.LogInfo(...) for Go, or LOG_INFO for C++. Service Extensions exports these log messages to Cloud Logging. You can log request and response headers and any actions that the plugin has taken.

  You can view these messages by using the networkservices.googleapis.com service.

• Media CDN log messages

  Media CDN log messages for HTTP requests to EdgeCacheService resources with an associated CdnEdgeExtension. These are regular Media CDN log entries that you can view under the edgecache.googleapis.com service.

Enable logging for a plugin

Service Extensions supports generating log messages during the execution of your plugin. Recording logs is disabled by default.

To record logs for a plugin, enable it when you create the plugin or update it.

To enable logging for an existing plugin, use the gcloud beta service-extensions wasm-plugins update command:

gcloud beta service-extensions wasm-plugins update WASM_PLUGIN \
    --log-config=[LOG_CONFIG,...]

Replace the following:

• WASM_PLUGIN: the ID or the fully qualified name of the plugin

• LOG_CONFIG: logging options for the plugin. To enable logging, set the enable option to true. Then, specify the following details:

  • sample-rate: the sampling rate of activity logs as a value between 0 and 1. The value 0 indicates that log messages aren't stored. The default value 1 indicates that all log messages are stored. A floating point value between 0.0 and 1.0 indicates that a percentage of log messages is stored.
  • min-log-level: the minimum severity level of plugin log messages to be exported to Cloud Logging. The default value is INFO.

After you enable logging for the plugin, you can view the messages emitted by logging statements in plugin code in Cloud Logging.

To view logs, in the Google Cloud console, go to the Logs Explorer page.

View log messages

Logs can be viewed by building queries in the Logs Explorer.

You can view plugin logs as standalone Service Extensions logs. In this view, each plugin log message is recorded in its own log record and isn't automatically associated with request log information.

These log messages are in the networkactions.googleapis.com/wasm_plugin_activity log with the resource type networkactions.googleapis.com/WasmPluginVersion.

The system might also add informational log messages to this log. For example, if there's a plugin failure when a plugin invocation exceeds CPU or memory limits, a message of ERROR severity is logged. Such messages can also be seen in View and filter errors.

You can also view log messages correlated with the corresponding Media CDN request logs in the Logs Explorer. While viewing request logs, when you expand a request log entry, the plugin logs related to the request appear nested under the request log. This view helps you understand plugin logs in the context of their associated request.

Log samples

Consider a sample Service Extensions log entry. The value of message is passed to the plugin's LOG_INFO call. The severity value depends on the log level used in the plugin log call. In the labels section, the value of the API is HTTP_REQUEST_HEADER, which indicates that the logged operation is the on_http_request_headers plugin callback.

{
  "insertId": "65224aac-0000-24bd-a0e1-582429bd544c@a1",
  "jsonPayload": {
    "@type": "type.googleapis.com/google.cloud.networkactions.logging.v1.WasmPluginLogEntry",
    "metroIataCode": "ber",
    "proxyRegionCode": "DE",
    "message": "[add_header_plugin.cc:26]::onRequestHeaders() AddHeaderStreamContext::onRequestHeaders called",
    "requestId": "effc0311-6716-431b-9e2a-7586835fdff1"
  },
  "resource": {
    "type": "networkactions.googleapis.com/WasmPluginVersion",
    "labels": {
      "plugin_version": "prod-1",
      "resource_container": "projects/123456789",
      "location": "global",
      "plugin_name": "add-headers-plugin-prod-resource"
    }
  },
  "timestamp": "2023-05-10T03:05:43.317015458Z",
  "severity": "INFO",
  "labels": {
    "networkactions.googleapis.com/operation": "HTTP_REQUEST_HEADERS"
  },
  "logName": "projects/123456789/logs/networkactions.googleapis.com%2Fwasm_plugin_activity",
  "trace": "projects/123456789/traces/effc0311-6716-431b-9e2a-7586835fdff1",
  "receiveTimestamp": "2023-05-10T03:05:44.207265284Z"
}

The related Media CDN log entry appears as follows. The requestId and trace values in both log messages are the same.

{
  "insertId": "6c95cc54-0000-26ba-ba73-f403043c328c@a1",
  "jsonPayload": {
    "cacheMode": "USE_ORIGIN_HEADERS",
    "@type": "type.googleapis.com/google.cloud.edgecache.v1.EdgeCacheLogEntry",
        "serviceExtensionInfo": [
      {
        "serviceExtension": "projects/123456789/locations/global/cdnEdgeExtensions/my-cdn-edge-extension",
        "chain": "edge-plugin-chain",
        "extension": "auth-and-header-plugin",
        "grpcStatus": "OK",
        "processingRequests": [
          {
            "eventType": "RESPONSE_HEADERS",
            "processingEffect": "CONTENT_MODIFIED"
          }
        ]
      }
    ],
    "cacheKeyFingerprint": "f04d02bb12f2e79e",
    "proxyStatus": "Google-Edge-Cache",
    "clientCity": "Erlangen",
    "clientAsn": "8881",
    "origin": "example-origin",
    "cacheId": "ber",
    "tlsVersion": "NONE",
    "latency": "0.002840304s",
    "originIp": "142.250.145.128",
    "requestId": "effc0311-6716-431b-9e2a-7586835fdff1",
    "clientRegionCode": "DE",
    "originalRequestId": "7656c418-df7c-4ae5-9db5-4f247ec5a82c",
    "httpTtfb": "0.002814903s",
    "metroIataCode": "BER",
    "proxyRegionCode": "DE",
    "cacheStatus": "hit"
  },
  "httpRequest": {
    "requestMethod": "GET",
    "requestUrl": "https://example.com/image.jpg",
    "requestSize": "1631",
    "status": "200",
    "responseSize": "402907",
    "userAgent": "Wtrace",
    "remoteIp": "2001:1438:1:19::5",
    "protocol": "HTTP/1.1"
  },
  "resource": {
    "type": "edgecache.googleapis.com/EdgeCacheRouteRule",
    "labels": {
      "location": "global",
      "route_destination": "projects/123456789/locations/global/edgeCacheOrigins/example-origin",
      "matched_path": "/turing/",
      "route_type": "ORIGIN",
      "path_matcher_name": "routes",
      "resource_container": "projects/123456789",
      "service_name": "example-service"
    }
  },
  "timestamp": "2023-05-10T03:05:43.321347304Z",
  "logName": "projects/bdn-prober/logs/edgecache.googleapis.com%2Fedge_cache_request",
  "trace": "projects/1069774196212/traces/effc0311-6716-431b-9e2a-7586835fdff1",
  "receiveTimestamp": "2023-05-10T03:05:49.919781008Z"
}

Limitations

Plugins are limited to logging up to 16 KiB of payload data per client HTTP request. This amount is divided across multiple logging calls that are associated with a given HTTP request. The limit applies only to log message text, not to additional metadata added to the log record by Service Extensions.

For example, if an on_http_request_headers callback makes two logging calls with 4 KiB messages each and then an on_http_response_headers callback tries to make three logging calls with 4 KiB messages each for the same HTTP request, the fifth logging message is dropped. A log message is added to record the number of plugin-generated log messages that were dropped.

Monitoring

This section describes how you can use Cloud Monitoring dashboards to view Service Extensions metrics for Media CDN plugins.

For detailed information about Service Extensions metric types, see the Google Cloud metrics page.

View the Monitoring dashboard for Service Extensions

To view the Monitoring dashboard for Service Extensions, do the following:

1. In the Google Cloud console, go to the Service Extensions page.

   Go to Service Extensions

2. Click the Plugins tab.
3. Click a plugin name.
4. On the Plugin details page, click the Monitoring tab.

On the Monitoring page, the metrics charts show information that can help you monitor plugin performance.

5. To view the metrics for plugin lifecycle operations, select values from the Operation filter list. By default, the values HTTP request header and HTTP response header are selected.
6. To view the metrics for a specific plugin version, select a value from the Plugin version filter list. By default, metrics are displayed for all versions.
7. To change the period for which you want to view the data, either select a predefined period from the time selector or click Custom and define a start and end time. By default, the selector is set to 1 day.

Plugin metrics for Service Extensions

You can monitor the following metrics for plugins from the Service Extensions perspective. These metrics have the prefix networkactions.googleapis.com/wasm_plugin/. The prefix is omitted from the entries in the table.

Metric type	Display name Kind, Type, Unit Description
invocation_count	Wasm plugin invocation count DELTA, INT64, 1 The number of invocations of the plugin over the selected time span. Each plugin callback invocation counts as a separate plugin invocation.
invocation_latencies	Wasm plugin invocation latency DELTA, DISTRIBUTION, us The local execution time, in milliseconds, of the plugin. The metric includes label-delineated entries for each callback.
cpu/usage_times	Wasm plugin normalized CPU usage DELTA, DISTRIBUTION, us{CPU} The CPU usage time of plugin invocations, in microseconds.
memory/bytes_used	Wasm plugin memory usage GAUGE, DISTRIBUTION, By Total memory allocated by Wasm plugin VMs, in bytes.

Service extension monitoring metrics for Media CDN

You can monitor Media CDN edge extensions from the EdgeCacheService perspective in Monitoring Metrics Explorer:

1. In the Google Cloud console, go to the Metrics explorer page.

   Go to Metrics Explorer

2. Select the Media CDN Service Extension monitored resource.

3. Select the Extension metric category.

4. Filter by the extension_service label set to your plugin name.

In Preview, you can monitor the following metrics for extensions on Media CDN. These metrics have the prefix networkactions.googleapis.com. The prefix is omitted from the entries in the following table.

The following table provides the metric type, display name, kind, type, unit, and description for each metric.

Metric type	Display name Kind, Type, Unit Description
extension/invocation_count	Extension invocation count DELTA, INT64, 1 The number of invocations sent to the extension.
extension/invocation_latencies	Extension invocation latencies DELTA, DISTRIBUTION, ms The distribution calculated from the latency of each extension invocation. Applicable only for callout extensions; not supported for plugins.
extension/sent_chunks_count	Extension sent chunks count DELTA, INT64, 1 Applicable only for request_body and response_body events. The number of data chunks sent to the extension.
extension/received_chunks_count	Extension received chunks count DELTA, INT64, 1 Applicable only for request_body and response_body events. The number of chunks received from the extension.
extension/failed_open_count	Extension failed invocations with fail-open DELTA, INT64, 1 The number of times that an invocation failed when the system was configured to fail open and the request was allowed to proceed.
extension/mutation_rejections_count	Extension mutation rejections count DELTA, INT64, 1 The number of invocations that requested header, body, or trailer mutations but were rejected. Rejections can occur for a variety of reasons, such as when the mutation is invalid or exceeds size limits.
extension/sent_bytes_count	Extension sent bytes count DELTA, INT64, By The number of bytes sent to the extension.
extension/received_bytes_count	Extension received bytes count DELTA, INT64, By The number of bytes received from the extension.