如需完成沿袭用例教程,请执行以下设置步骤:
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
-
Create a project: To create a project, you need the Project Creator role
(
roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.createpermission. Learn how to grant roles.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
-
Create a project: To create a project, you need the Project Creator role
(
roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.createpermission. Learn how to grant roles.
-
Enable the BigQuery, Data lineage , Dataform, BigQuery Data Transfer, and Secret Manager APIs.
Roles required to enable APIs
To enable APIs, you need the
serviceusage.services.enablepermission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.For new projects, the BigQuery API is automatically enabled.
所需角色
如需获得执行沿袭用例教程所需的权限,请让您的管理员为您授予项目的以下 IAM 角色:
- Data Lineage Viewer (
roles/datalineage.viewer):针对记录和查看沿袭的项目。 - BigQuery Data Viewer (
roles/bigquery.dataViewer):针对表的存储项目。 - BigQuery Resource Viewer (
roles/bigquery.resourceViewer):针对作业的计算项目。 - Dataplex Catalog Viewer (
roles/dataplex.catalogViewer):针对存储目录条目的项目。 - Dataform Editor (
roles/dataform.editor):针对工作区和代码库所在的项目。
以下列表介绍了与所需角色关联的项目类型和服务:
- 存储项目用于存储 BigQuery 数据集和表。
- 计算项目用于处理数据和存储沿袭元数据。它用于运行 BigQuery 作业和数据转换。
- 目录条目包含描述表的元数据,让您无需访问底层数据即可查找和整理表。
- 沿袭项目用于记录和可视化呈现数据及其转换的历史记录。
- Dataform 是一种用于构建、版本控制和运行基于 SQL 的数据流水线的服务。它将原始数据转换为整洁有序且有文档记录的数据集。
如需详细了解如何授予角色,请参阅管理对项目、文件夹和组织的访问权限。您也可以通过自定义角色或其他预定义角色来获取所需的权限。
开始使用
如需完成教程,请使用 Data Lineage Use Cases 代码库。此代码库包含用于设置数据集和运行数据转换的预定义代码。
数据集概览
每个教程都使用不同的真实数据集(例如医疗服务提供方、就业或商家数据)来演示实际场景中的数据沿袭。
设置数据集
如需使用数据沿袭跟踪数据更改,请执行以下一次性设置:
- 创建个人访问令牌并将其存储在 Secret Manager 中。
- 将代码库关联到 Dataform。
设置完成后,运行数据转换以处理数据并生成沿袭。