Create observability buckets

You can manually create the _Trace observability bucket before your project receives trace data to customize its storage location and customer-managed encryption key (CMEK) settings. If your Google Cloud project ingests trace data before this bucket exists, then Google Cloud Observability automatically provisions the bucket using your project's default settings for observability buckets.

For information about how Google Cloud Observability stores data, see Storage overview.

Interaction with organization policies

A request to create an observability bucket verifies that the command parameters comply with organization policies. For example, if an organization policy restricts resource locations, then creating a bucket fails if you specify a restricted location.

Interaction with default settings for observability buckets

When Google Cloud Observability automatically creates an observability bucket due to ingestion of data, it uses the default settings for observability buckets that apply to the parent resource of the bucket. Those default settings might be defined in the parent or in a hierarchical ancestor of the parent, and they specify the following:

• The storage location.
• The Cloud KMS key to use for the stored data.

When creating an observability bucket, you must specify a location. Google Cloud Observability applies the Cloud KMS key defined in your default settings unless you explicitly specify a different key in your create request.

You cannot create a bucket with Google-default encryption if the applicable default settings specify a Cloud KMS key. To use Google-default encryption, ensure no Cloud KMS key is configured in your default settings.

For information about default settings for observability buckets, see Set defaults for observability buckets.

Limitations

The following restrictions apply:

• You must specify a supported location.
• The BUCKET_ID must be _Trace.
• The display name must not exceed 100 encoded bytes.
• The description must not exceed 1000 encoded bytes.
• Data is stored for 30 days. You must either omit the retention period or set it to 30.
• If you provide a Cloud KMS key, then the location of the key must exactly match the parent location of the observability bucket.
• You can only create observability buckets in Google Cloud projects.
• A Google Cloud project can have at most one observability bucket named _Trace.

Before you begin

Configure your project and your IAM roles, and select the interface that you plan to use.

Configure your project and roles

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Observability API.

Roles required to enable APIs

To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

Enable the API

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Observability API.

Roles required to enable APIs

To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

Enable the API

1. To get the permissions that you need to create observability buckets, ask your administrator to grant you the Observability Editor (roles/observability.editor) IAM role on your project. For more information about granting roles, see Manage access to projects, folders, and organizations.

   You might also be able to get the required permissions through custom roles or other predefined roles.

Configure interfaces

gcloud

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

REST

To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

Install the Google Cloud CLI.

If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

Configure Cloud KMS key

Optional. If you plan to create an observability bucket and to specify a Cloud KMS key, then do the following:

1. Enable the Cloud Key Management Service API.

   Roles required to enable APIs

   To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

   Enable the API

2. Create a key ring and keys.

   The location of the observability bucket must match the location of the key.

3. Replace PROJECT_ID with your project's ID, and then run the following command:

   gcloud beta observability settings describe \
   --location=global --project=PROJECT_ID
   

   The preceding command checks whether you've configured a default storage location. It also creates the Google Cloud Observability service account when that account doesn't exist. The command's response lists the service account's ID.

4. Grant the Cloud KMS CryptoKey Encrypter/Decrypter role to the Google Cloud Observability service account.

   gcloud kms keys add-iam-policy-binding \
   --project=KMS_PROJECT_ID \
   --member=serviceAccount:[e-mail] \
   --role=roles/cloudkms.cryptoKeyEncrypterDecrypter \
   --location=KMS_KEY_LOCATION \
   --keyring=KMS_KEY_RING \
   KMS_KEY_NAME
   

   Before running the preceding command, make the following replacements:

   • KMS_PROJECT_ID: The unique alphanumeric identifier, composed of your Google Cloud project name and a randomly assigned number, of the Google Cloud project running Cloud KMS. For information about getting this identifier, see Identifying projects.
   • service-PROJECT_NUMBER: The name of the Observability service account that was listed in the response of the preceding step.
   • KMS_KEY_LOCATION: The Cloud KMS key's region.
   • KMS_KEY_RING: The Cloud KMS key ring's name.
   • KMS_KEY_NAME: The Cloud KMS key's name. It is formatted like this: projects/KMS_PROJECT_ID/locations/LOCATION/keyRings/KMS_KEY_RING/cryptoKeys/KEY.

Create an observability bucket

REST

To create an observability bucket, send a request to projects.locations.buckets.create.

You must specify the parent parameter, which has the following form:

projects/PROJECT_ID/locations/LOCATION

The fields in the preceding expression have the following meanings:

• PROJECT_ID: The identifier of the project.
• LOCATION: The location of the observability bucket.

The request body is a Bucket object. Complete the following fields:

• name: Set this field to the following:

  projects/PROJECT_ID/locations/LOCATION/buckets/_Trace
  

• Optional: Provide values for the displayName and description fields.

• Optional: Provide a CMEK. When specified, this key encrypts the stored data.

  If you don't provide a CMEK, then the default settings that apply to the bucket's parent resource determine the encryption key. If the default settings specify a Cloud KMS key, then that key encrypts the stored data. Otherwise, Google-default encryption is used.

The response is an Operation object. Poll the projects.locations.operations.get method until the Operation.done field is set to true. Other fields in the Operation structure provide information about the success or failure of the request.

List observability buckets

You can list the observability buckets to verify that the create request completed successfully.

This section describes how to list your observability buckets. An observability bucket is the management entity for datasets, which store data.

gcloud

Note: You must use gcloud CLI version 563.0.0 or later.

Before using any of the command data below, make the following replacements:

• LOCATION: The location of the observability buckets. To list all observability buckets, regardless of location, set the location to a hyphen (-).
• PROJECT_ID: The identifier of the project..

Execute the gcloud beta observability buckets list command:

Linux, macOS, or Cloud Shell

gcloud beta observability buckets list \
 --location=LOCATION --project=PROJECT_ID

Windows (PowerShell)

gcloud beta observability buckets list `
 --location=LOCATION --project=PROJECT_ID

Windows (cmd.exe)

gcloud beta observability buckets list ^
 --location=LOCATION --project=PROJECT_ID

The response lists the name, description, and create time of each observability buckets. The following is an example of a response when the command is successful:

---
createTime: '2026-01-21T21:39:22.381083860Z'
description: Bucket for storing spans from Cloud Trace.
name: projects/my-project/locations/us/buckets/_Trace

REST

To list the observability buckets that are in your project and in a specific location, use the projects.locations.buckets.list method.

You must specify the parent parameter, which has the following form:

projects/PROJECT_ID/locations/LOCATION

The fields in the previous expression have the following meanings:

• PROJECT_ID: The identifier of the project.
• LOCATION: The location of the observability bucket. If you set LOCATION to a hyphen, (-), then all observability buckets in your project are listed.

The response is an array of Bucket objects. For each object, the value of the name field has the following format:

projects/PROJECT_ID/locations/LOCATION/buckets/BUCKET_ID

The following shows a sample response:

{
  "buckets": [
    {
      "name": "projects/my-project/locations/us/buckets/_Trace",
      "description": "Trace Bucket",
      "createTime": "2025-01-01T15:42:30.988919645Z",
      "updateTime": "2025-02-04T15:42:30.988919645Z",
      "retentionDays": 30
    }
  ]
}

You can use the Observability API to get more information about the bucket whose ID is BUCKET_ID. For example, you can list the datasets on that bucket, and the views and links on each dataset. For more information, see the Observability API reference documentation.

What's next

• View and analyze telemetry

• Set defaults for observability buckets: Configure a default storage location and a default customer-managed encryption key (CMEK) for your observability buckets.