Trace storage overview

Your Cloud Trace data is stored in an observability dataset managed by an observability bucket. Understanding the storage model helps you control where your trace data is stored, enforce encryption policies, and connect telemetry to analytics services provided by Observability Analytics and BigQuery.

For information about how to use BigQuery to analyze your trace data, see Create a linked BigQuery dataset.

Observability storage model

The Observability API storage model relies on the following architecture:

Observability buckets

An observability bucket is the management entity for datasets, which store data. An observability bucket is in a specific location and has a data retention policy. When a Google Cloud service uses the Observability API to store their data, the system creates an observability bucket based on the name of the service, a dataset to store data, and a view to provide read access to the stored data. For example, for the Cloud Trace service, the system names the system-created bucket _Trace, the dataset Spans, and the view _AllSpans. To learn about the structure of an observability bucket, see Bucket.

Datasets

A dataset stores data. The system automatically creates one dataset when it creates an observability bucket, which manages the dataset. For example, when the system creates the _Trace bucket, it also creates a dataset named Spans, which stores your trace data. To learn about the structure of a dataset, see Dataset.

Views on datasets

Each dataset hosts one or more view. A view provides read access to a subset of entries in the dataset. The system creates one view when it creates a dataset. That view includes all data in the dataset. The name of the view depends on the service. For example, for the Cloud Trace service, the system creates a view named _AllSpans on the Spans dataset. To learn about the structure of a view, see View.

Links on datasets

Each dataset can contain at most one link. When you create a link for a dataset, the system creates a linked BigQuery dataset. You can then query the data in your dataset using BigQuery or by other services that use the BigQuery API. To learn about the structure of a link, see Link.

The system doesn't automatically create links on datasets.

Storage configuration for trace data

Trace data is stored in a dataset that is managed by the observability bucket named _Trace. To hold the dataset, the _Trace bucket must exist. The _Trace bucket can be created automatically or manually:

Automatic creation: The system automatically creates the bucket in response to receiving trace data from an application or a Google Cloud service. The system uses the applicable default settings for observability buckets to determine the bucket's location and encryption key. If you haven't defined default settings, then the system selects a supported location and the bucket uses Google-default encryption.

Trace data generated by Cloud Run functions, Cloud Run, and App Engine doesn't cause the system to create the observability bucket. Spans from these services are stored only when the observability bucket exists.
Manual creation: You can use the Observability API to create the _Trace bucket before your Google Cloud project receives trace data. You must provide the location of the bucket. You can provide a Cloud Key Management Service key:
- If you provide a key, then the system uses that key to encrypt the stored data.
- If you don't provide a Cloud KMS key, then the default settings that apply to the bucket's parent resource determine the encryption key. If the default settings specify a Cloud KMS key, then that key encrypts the stored data. Otherwise, Google-default encryption is used.

When the _Trace bucket is created, the system also creates a dataset named Spans for the bucket, and a view named _AllSpans on the dataset. That view includes all data in the dataset.

For more information, see the following:

If trace data appears on the Trace Explorer page, then your observability bucket named _Trace exists. If you don't see any data or if you see a banner stating that storage isn't initialized, then try one of the following:

Data residency for observability buckets

If you have compliance or regulatory requirements to store your data in specific locations or to use customer-managed encryption keys (CMEKs), then we recommend that you configure both organization policies and default settings for observability buckets.

For organizations, folders, and projects, default settings for observability buckets let you configure the following:

A default storage location.
For each location, a default Cloud Key Management Service key.

There are two different ways the system uses default settings:

When the system automatically creates an observability bucket, it uses the default settings to determine the bucket's location and encryption key. If you haven't defined default settings, then the system selects the location and the bucket uses Google-default encryption.
When you initiate creation of an observability bucket by issuing an API request, you provide the location. However, the system automatically uses the Cloud KMS key defined by the default settings to encrypt the data, unless the arguments to the API request specify the key.

Because the parent of an observability bucket is always a project, when you create a bucket, the system first searches for project-level default settings. If those don't exist, then the system searches the parent's ancestors for default settings. For example, if you define default settings for a folder, then those settings apply to descendants of the folder, except those descendants that have default settings configured.

You can also use organization policies to restrict the locations of new observability buckets, to require the use of CMEKs, or to restrict which Cloud KMS keys can be used for encryption. If you configure organization policies that require use of CMEKs, then you must configure default settings for observability buckets. If you don't, then provisioning for system-created observability buckets fails.

For more information, see Set defaults for observability buckets.

Limitations

You can't do the following:

Modify or delete observability buckets.
Create, delete, or modify datasets.
Create, delete, or modify views.
Use the Google Cloud console to list buckets, datasets, views, or links.