To complete the lineage use case tutorials, perform the following setup steps:

1.


   Enable the BigQuery, Data lineage, Dataform, BigQuery Data Transfer, and Secret Manager APIs, if any are not already enabled.


   **Roles required to enable APIs**


   To enable APIs, you need the `serviceusage.services.enable` permission. If you
   created the project, then you likely already have this permission through the
   Owner role (`roles/owner`). Otherwise, you can get this permission through the
   Service Usage Admin role (`roles/serviceusage.serviceUsageAdmin`).
   [Learn how to grant roles](https://docs.cloud.google.com/iam/docs/granting-changing-revoking-access.md.txt).

   [Enable the APIs](https://console.cloud.google.com/apis/enableflow?apiid=bigquery.googleapis.com,datalineage.googleapis.com,dataform.googleapis.com,bigquerydatatransfer.googleapis.com,secretmanager.googleapis.com)

   For new projects, the BigQuery API is automatically enabled.

## Required roles

To get the permissions that you need to perform the lineage use case tutorials, ask your administrator to grant you the following IAM roles on your projects:

- **Data Lineage Viewer** (`roles/datalineage.viewer`): on the project where lineage is recorded and viewed.
- **BigQuery Data Viewer** (`roles/bigquery.dataViewer`): on the table's storage project.
- **BigQuery Resource Viewer** (`roles/bigquery.resourceViewer`): on the job's compute project.
- **Dataplex Catalog Viewer** (`roles/dataplex.catalogViewer`): on the project where catalog entries are stored.
- **Dataform Editor** (`roles/dataform.editor`): on the project where your workspaces and repositories are located.

The following list describes the project types and services associated with the required roles:

- Storage project stores the BigQuery datasets and tables.
- Compute project processes your data and stores the lineage metadata. It is used to run BigQuery jobs and data transformations.
- Catalog entries contain metadata that describes your tables, allowing you to find and organize them without accessing the underlying data.
- Lineage project records and visualizes the history of your data and its transformations.
- Dataform is a service used to build, version control, and run SQL-based data pipelines. It transforms raw data into clean, documented datasets.

For more information about granting roles, see [Manage access to projects, folders, and organizations](https://docs.cloud.google.com/iam/docs/granting-changing-revoking-access.md.txt). You might also be able to get the required permissions through [custom roles](https://docs.cloud.google.com/iam/docs/creating-custom-roles.md.txt) or other [predefined roles](https://docs.cloud.google.com/iam/docs/roles-overview.md.txt#predefined).

## Get started

To complete the tutorials, use the [Data lineage use cases repository](https://github.com/GoogleCloudPlatform/dataplex-labs/tree/main/lineage-uses-cases-documentation). This repository contains predefined code to set up datasets and run data transformations.

### Datasets overview

Each tutorial uses a different real-world dataset, such as healthcare provider, employment, or business data, to demonstrate data lineage in realistic scenarios.

### Set up datasets

To track data changes with data lineage, perform this one-time setup:

1. Create a personal access token and store it in Secret Manager.
2. Link the repository to Dataform.

After setup, run the data transformations to process the data and generate lineage.