Structured logging

This document discusses the concept of structured logging and the methods for adding structure to log entry payload fields. When the log payload is formatted as a JSON object and that object is stored in the jsonPayload field, the log entry is called a structured log. For these logs, you can construct queries that search specific JSON paths and you can index specific fields in the log payload. In contrast, when the log payload is formatted as a string and stored in the textPayload field, the log entry is unstructured. You can search the text field, but you can't index its content.

To create structured log entries, do any of the following:

  • Call the entries.write API method and supply a fully formatted LogEntry.
  • Use the gcloud logging write command.
  • Use a Cloud Logging client library which writes structured logs.
  • Use the BindPlane service.

  • Use an agent to write logs:

    • Some Google Cloud services contain an integrated logging agent that sends the data written to stdout or stderr as logs to Cloud Logging. You can use this approach for Google Cloud services such as Google Kubernetes Engine, App Engine flexible environment, and Cloud Run functions.
    • For Compute Engine virtual machines (VMs), we recommend that you install and configure the Ops Agent to send structured logs to Cloud Logging. If your existing VMs use the legacy Logging agent, note that the agent has reached its end of support. For information about migration alternatives from the legacy Logging agent, see the migration guide, or see Structured logging with the legacy Logging agent for information about using the legacy Logging agent to write structured logs.

For code examples, see Writing structured logs.

Write logs by using the gcloud CLI

You can write log data by using the gcloud CLI. The interface supports unstructured logs and structured logs. When you want to write a structured log, provide the command a serialized JSON object.

For a quickstart, see Write and query log entries with the Google Cloud CLI.

For code examples, see the gcloud logging write reference.

Write logs by using BindPlane

You can use the BindPlane service to send logs to Logging. For these logs, the payloads are in JSON format and are structured according to the source system. For information on finding and viewing logs ingested by using BindPlane, see the BindPlane Quickstart Guide.

Write logs by using the Ops Agent

To get logs from your Compute Engine instances, you can use the Ops Agent. The Ops Agent combines logging and metrics into a single agent, provides a YAML-based configuration, and features high-throughput logging.

For information about how to configure the Ops Agent to support structured logging or to customize the form of a structured log, see Configure the Ops Agent.

Troubleshoot issues

To troubleshoot common issues found with installing or interacting with the Ops Agent, see Troubleshooting the agent.

What's next