Trace storage overview

This document describes the storage model for Cloud Trace data. It introduces observability buckets, observability views, and links. If you create a link, then you can query your trace data from BigQuery.

This document doesn't describe the schema, which specifies how individual spans are stored. To learn more, see Trace schema.

Observability storage model

The Observability API storage model relies on the following architecture:

Observability buckets
An observability bucket is the management entity for datasets, which store data. An observability bucket is in a specific location and has a data retention policy. When a Google Cloud service uses the Observability API to store their data, the system creates an observability bucket based on the name of the service. For example, for the Cloud Trace service, the system-created bucket is named _Trace. To learn about the structure of an observability bucket, see Bucket.
Datasets
A dataset is a storage entity. Each dataset is a child of an observability bucket. When the system creates an observability bucket for a Google Cloud service, it also creates one dataset. For example, after the system creates the _Trace bucket, it creates the dataset named Spans. That dataset stores your trace data. To learn about the structure of a dataset, see Dataset.
Views on datasets
Each dataset hosts one or more view. A view provides read access to a subset of entries in the dataset. When a dataset is created, the system automatically creates one view. That view includes all data in the dataset. The name of the view depends on the service. For example, for the Cloud Trace service, the system creates a view named _AllSpans on the Spans dataset. To learn about the structure of a view, see View.
Links on datasets

Each dataset can contain at most one link. When you create a link for a dataset, the system creates a linked BigQuery dataset. You can then query the data in your dataset by using BigQuery or by other services that use the BigQuery API. To learn about the structure of a link, see Link.

The system doesn't automatically create links on datasets.

Storage configuration for trace data

Your trace data is stored in an observability bucket named _Trace. The system creates a dataset named Spans on this bucket. On that dataset, the system creates a view named _AllSpans. This view includes all data in the dataset.

The system makes a single attempt to create the bucket named _Trace when the bucket doesn't exist and either of the following are true:

  • An application sends trace data to your Google Cloud project by using the Cloud Trace API or the Telemetry API.

  • You enable Cloud Trace for a Google Cloud service, and then that service sends trace data to your Google Cloud project. For example, Cloud Service Mesh supports collecting traces; however, trace collection is disabled by default. If you enable tracing for Cloud Service Mesh, then the data sent by that service causes the system to create the bucket.

    Trace data generated by Cloud Run functions, Cloud Run, and App Engine don't cause the system to create the observability bucket. Spans from these services are stored only when the observability bucket exists.

To determine whether the observability bucket named _Trace exists or to trigger a flow that creates this bucket, do the following:

  1. In the Google Cloud console, go to the Trace explorer page:

    Go to Trace explorer

    You can also find this page by using the search bar.

  2. Do one of the following:

    • If the page displays trace data, then you don't need to do anything. Your Google Cloud project contains an observability bucket named _Trace.

    • If the page displays the following banner, then your doesn't have any trace data, or it only has data from Cloud Run functions, Cloud Run, and App Engine.

      Trace storage is not initialized for this project. Enable trace storage to begin collecting trace data.

      If you want to trigger a flow that creates an observability bucket named _Trace, then go to the banner and click Enable.

    • If the page displays the following banner, then initialization of the observability bucket named _Trace failed. If you this message, then go to the banner and click File a ticket.

      Initializing trace storage has failed for an unexpected reason. Please file a support ticket for assistance.

Limitations

You can't do the following:

  • Modify or delete observability buckets.
  • Create, delete, or modify datasets.
  • Create, delete, or modify views.
  • Use the Google Cloud console to list buckets, datasets, views, or links.
  • Create observability buckets in the locations other than the us; an observability bucket must be in the us location.

What's next