Trace storage overview

This document describes the storage model for Cloud Trace data. It introduces observability buckets, observability views, and links. If you create a link, then you can query your trace data from BigQuery.

This document doesn't describe the schema, which specifies how individual spans are stored. To learn more, see Trace schema.

Observability storage model

The Observability API storage model relies on the following architecture:

Observability buckets
An observability bucket is the management entity for datasets, which store data. An observability bucket is in a specific location and has a data retention policy. When a Google Cloud service uses the Observability API to store their data, the system creates an observability bucket based on the name of the service, a dataset to store data, and a view to provide read access to the stored data. For example, for the Cloud Trace service, the system names the system-created bucket _Trace, the dataset Spans, and the view _AllSpans. To learn about the structure of an observability bucket, see Bucket.
Datasets
A dataset stores data. The system automatically creates one dataset when it creates an observability bucket, which manages the dataset. For example, when the system creates the _Trace bucket, it also creates a dataset named Spans, which stores your trace data. To learn about the structure of a dataset, see Dataset.
Views on datasets
Each dataset hosts one or more view. A view provides read access to a subset of entries in the dataset. The system creates one view when it creates a dataset. That view includes all data in the dataset. The name of the view depends on the service. For example, for the Cloud Trace service, the system creates a view named _AllSpans on the Spans dataset. To learn about the structure of a view, see View.
Links on datasets

Each dataset can contain at most one link. When you create a link for a dataset, the system creates a linked BigQuery dataset. You can then query the data in your dataset by using BigQuery or by other services that use the BigQuery API. To learn about the structure of a link, see Link.

The system doesn't automatically create links on datasets.

Storage configuration for trace data

Your trace data is stored in an observability bucket named _Trace. The system creates a dataset named Spans on this bucket. On that dataset, the system creates a view named _AllSpans. This view includes all data in the dataset.

The system tries to create the observability bucket named _Trace when the bucket doesn't exist and either of the following are true:

  • An application sends trace data to your Google Cloud project by using the Cloud Trace API or the Telemetry API.

  • You enable Cloud Trace for a Google Cloud service, and then that service sends trace data to your Google Cloud project. For example, Cloud Service Mesh supports collecting traces; however, trace collection is disabled by default. If you enable tracing for Cloud Service Mesh, then the data sent by that service causes the system to create the bucket.

    Trace data generated by Cloud Run functions, Cloud Run, and App Engine don't cause the system to create the observability bucket. Spans from these services are stored only when the observability bucket exists.

If you can view trace data by using the Trace Explorer page, then your observability bucket named _Trace exists. If you don't see any data or if you see a banner that states that storage isn't initialized, then try one of the of following:

Data residency for observability buckets

If you have compliance or regulatory requirements to store your data in specific locations or to use CMEK, then we recommend that you configure default settings for observability buckets:

  • For organizations, folders, and projects, default settings for observability buckets let you configure the following:

    • A default storage location.
    • For each location, a default Cloud Key Management Service key.

    Descendants in the resource hierarchy automatically use these settings, except for those descendants where you've configured default settings.

    The default settings for observability buckets apply only to new resources, not to existing resources. To learn more, see Set defaults for observability buckets.

The default settings for observability buckets settings don't apply to log buckets, which store log data. To learn how to set the default location or require CMEK for log buckets, see Configure default resource settings for Cloud Logging.

Limitations

You can't do the following:

  • Modify or delete observability buckets.
  • Create, delete, or modify datasets.
  • Create, delete, or modify views.
  • Use the Google Cloud console to list buckets, datasets, views, or links.

What's next