A linked [BigQuery dataset](https://docs.cloud.google.com/bigquery/docs/analytics-hub-introduction.md.txt#linked_datasets) on your observability dataset
lets you query Cloud Trace data by using BigQuery services.
You can then join trace data with business datasets and analyze spans by using
BigQuery Studio or the BigQuery API.

If you use Observability Analytics, then you can query trace data with SQL and join your
trace and log data without creating a linked BigQuery dataset.
For more information, see [Query and analyze traces](https://docs.cloud.google.com/trace/docs/analytics.md.txt)
and [Query and analyze logs](https://docs.cloud.google.com/logging/docs/analyze/query-and-view.md.txt).

## When you need a linked dataset

You need a linked BigQuery dataset when you
want to do any of the following:

- Join trace data with other BigQuery datasets.
- Query trace data from another service like BigQuery Studio or Data Studio.
- [Run queries from the **Observability Analytics** page on the BigQuery engine](https://docs.cloud.google.com/trace/docs/analytics-run-on-bigquery-engine.md.txt).

  The **Observability Analytics** page automatically runs queries on the default query engine,
  which uses free slots. However, you can configure this page to run queries
  on the BigQuery engine.
- [Monitor your SQL query results with an alerting policy](https://docs.cloud.google.com/trace/docs/sql-in-alerting.md.txt).

- Run automated data pipelines, continuous analytical queries, or AI agent
  workflows that programmatically read high volumes of trace data without
  Trace API rate limits.

## Create a linked BigQuery dataset

This section describes how to create a linked BigQuery dataset on an
observability dataset, which lets you use BigQuery services to
query your trace data. Each observability dataset supports one linked
BigQuery dataset.

When you create a link on an observability dataset, the following occurs:

- Google Cloud Observability might create the following service accounts or modify their
  IAM role grants:

  - [Monitoring Service Account](https://docs.cloud.google.com/iam/docs/service-agents.md.txt#monitoring-service-account):
    Required to create a link to a linked BigQuery dataset.
    The Cloud Monitoring API must be enabled for Google Cloud Observability to create this
    service account.

  - [Cloud Observability Service Account](https://docs.cloud.google.com/iam/docs/service-agents.md.txt#cloud-observability-service-account):
    Required to create a link on an observability dataset.

  - [Cloud Logging Service Account](https://docs.cloud.google.com/iam/docs/service-agents.md.txt#cloud-logging-service-account):
    Required to create a link on a log bucket.

- Audit logs record the request to create a link and the request from the
  service agent manager to grant the Monitoring Service Account the
  IAM role of
  [Monitoring Service Agent](https://docs.cloud.google.com/iam/docs/roles-permissions/monitoring.md.txt#monitoring.notificationServiceAgent).
  These logs also record the completion of the long-running operation.

### Before you begin

1.

   To get the permissions that
   you need to create a link on an observability dataset,

   ask your administrator to grant you the
   [Observability Editor](https://docs.cloud.google.com/iam/docs/roles-permissions/observability.md.txt#observability.editor) (`roles/observability.editor`) IAM role on your project.


   For more information about granting roles, see [Manage access to projects, folders, and organizations](https://docs.cloud.google.com/iam/docs/granting-changing-revoking-access.md.txt).


   You might also be able to get
   the required permissions through [custom
   roles](https://docs.cloud.google.com/iam/docs/creating-custom-roles) or other [predefined
   roles](https://docs.cloud.google.com/iam/docs/roles-overview#predefined).

### Create dataset

### gcloud

<br />

> [!NOTE]
> **Note**: You must use gcloud CLI version 586.0.0 or later.

<br />


Before using any of the command data below,
make the following replacements:

- <var translate="no">LINK_ID</var>: The name of the BigQuery dataset.
- <var translate="no">DATASET_ID</var>: The ID of the dataset. Your trace data is stored in a dataset named `Spans`.
- <var translate="no">BUCKET_ID</var>: The ID of the observability bucket. For example, this ID might be `_Trace`.
- <var translate="no">LOCATION</var>: The location of the observability bucket.
- <var translate="no">PROJECT_ID</var>: The identifier of the project.


Execute the

[`gcloud observability buckets datasets links create`](https://docs.cloud.google.com/sdk/gcloud/reference/observability/buckets/datasets/links/create)

command:

#### Linux, macOS, or Cloud Shell

```bash
gcloud observability buckets datasets links create \
  projects/PROJECT_ID/locations/LOCATION/buckets/BUCKET_ID/datasets/DATASET_ID/links/LINK_ID \
 --dataset=DATASET_ID\
 --bucket=BUCKET_ID \
 --location=LOCATION \
 --project=PROJECT_ID
```

#### Windows (PowerShell)

```bash
gcloud observability buckets datasets links create `
  projects/PROJECT_ID/locations/LOCATION/buckets/BUCKET_ID/datasets/DATASET_ID/links/LINK_ID `
 --dataset=DATASET_ID`
 --bucket=BUCKET_ID `
 --location=LOCATION `
 --project=PROJECT_ID
```

#### Windows (cmd.exe)

```bash
gcloud observability buckets datasets links create ^
  projects/PROJECT_ID/locations/LOCATION/buckets/BUCKET_ID/datasets/DATASET_ID/links/LINK_ID ^
 --dataset=DATASET_ID^
 --bucket=BUCKET_ID ^
 --location=LOCATION ^
 --project=PROJECT_ID
```


The create command initiates a long-running operation. The following is
an example of a response when the command is successful:

```
Create request issued for: [mydataset]
Waiting for operation [projects/my-project/locations/us/operations/operation-1775164903749-64e80c9817833-9ff804b6-c3e9cbe7] to complete...done.
Created link [mydataset].
```

<br />

### Terraform

To create a link on an observability dataset, use the Terraform resource
[`google_observability_link`](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/observability_link)
and set the following fields:

- `project`: The ID of your project.
- `location`: The location of the link. This location must match the location of the observability bucket.
- `bucket`: The ID of the observability bucket. For example, `_Trace`.
- `dataset`: The ID of the observability dataset. For example, `Spans`.
- `link_id`: The ID of the BigQuery dataset.

You can also update and delete links.

### REST

To create a link to a BigQuery dataset, send a request to the
[`projects.locations.buckets.datasets.links.create`](https://docs.cloud.google.com/stackdriver/docs/reference/observability/api/rest/v1/projects.locations.buckets.datasets.links/create.md.txt)
endpoint.

You must specify the parent parameter, which has the following form:

    projects/PROJECT_ID/locations/LOCATION/buckets/BUCKET_ID/datasets/DATASET_ID

The fields in the preceding expression have the following meanings:

- <var translate="no">PROJECT_ID</var>: The identifier of the project.
- <var translate="no">LOCATION</var>: The [location of the observability bucket](https://docs.cloud.google.com/stackdriver/docs/observability/observability-bucket-locations.md.txt).
- <var translate="no">BUCKET_ID</var>: The ID of the observability bucket. For example, this ID might be `_Trace`.
- <var translate="no">DATASET_ID</var>: The ID of the dataset being queried. For example, this ID might be `Spans`.

This command requires a query parameter and a request body:

- The query parameter, `linkId`, must be specified and set to the name of the
  BigQuery dataset. For example, `linkId="my_link"`. The
  BigQuery dataset name must be unique for your Google Cloud project,
  and must be limited to 100 characters and can include only letters,
  digits, and underscores.

- The request body is a [`Link`](https://docs.cloud.google.com/stackdriver/docs/reference/observability/api/rest/v1/projects.locations.buckets.datasets.links.md.txt#Link) object. The value of the
  `name` field has the following format:

      projects/PROJECT_ID/locations/LOCATION/buckets/BUCKET_ID/datasets/DATASET_ID/links/LINK_ID

  The value you provide for the `name` field must match the
  linked BigQuery dataset referenced by the query parameter.

  The <var translate="no">LINK_ID</var> field is the name of the BigQuery dataset.

The response is an [`Operation`](https://docs.cloud.google.com/stackdriver/docs/reference/observability/api/rest/v1/folders.locations.operations.md.txt#Operation) object.
This object contains information about the progress of the method. When the
method completes, the `Operation` object contains status data.

For a complete list of Observability API endpoints,
see the [Observability API reference documentation](https://docs.cloud.google.com/stackdriver/docs/reference/observability/api/rest.md.txt).

If you encounter permission errors when creating a linked dataset,
then see [Troubleshoot permission errors](https://docs.cloud.google.com/trace/docs/troubleshooting.md.txt#linked-dataset-permission-error).

## What's next

- [Query trace data with BigQuery Studio](https://docs.cloud.google.com/trace/docs/analytics-query-with-bigquery-studio.md.txt)
- [Run queries on BigQuery engine](https://docs.cloud.google.com/trace/docs/analytics-run-on-bigquery-engine.md.txt)