您可以在專案、資料夾或組織層級,選擇性啟用或停用特定整合項目的歷程擷取功能。
如要進一步瞭解支援的整合和設定情境,請參閱「控管資料歷程擷取」。
必要條件
如要控管歷程擷取作業,請使用 Data Lineage API。Data Lineage API 是用戶端 API,因此請務必為用戶端專案設定計費和配額。
角色和權限
如要取得設定及控管資料歷程擷取作業所需的權限,請要求管理員授予您下列 Identity and Access Management (IAM) 角色:
如要取得設定及控管資料歷程擷取作業所需的權限,請要求管理員授予您專案的下列 IAM 角色:
-
設定及管理資料歷程:資料歷程管理員 (
roles/datalineage.admin)
如要進一步瞭解如何授予角色,請參閱「管理專案、資料夾和組織的存取權」。
這些預先定義的角色具備設定及控管資料歷程擷取作業所需的權限。如要查看確切的必要權限,請展開「Required permissions」(必要權限) 部分:
所需權限
您必須具備下列權限,才能設定及控管資料歷程擷取作業:
-
取得及更新歷程設定:
-
datalineage.configs.get -
datalineage.configs.update
-
在用戶端專案中啟用
datalineage.googleapis.comAPI。詳情請參閱「啟用資料歷程」。設定用戶端專案。在以下範例中,請使用
X-Goog-User-Project標頭。詳情請參閱「系統參數」。
取得目前設定
如要檢查資源是否已啟用歷程擷取功能,或在修改設定前取得 etag 值,請擷取目前的設定。
C#
C#
在試用這個範例之前,請先按照「使用用戶端程式庫的 Knowledge Catalog 快速入門導覽課程」中的 C# 設定說明操作。詳情請參閱 Knowledge Catalog C# API 參考文件。
如要向 Knowledge Catalog 進行驗證,請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。
Go
Go
在試用這個範例之前,請先按照「使用用戶端程式庫的 Knowledge Catalog 快速入門導覽課程」中的 Go 設定說明操作。詳情請參閱 Knowledge Catalog Go API 參考文件。
如要向 Knowledge Catalog 進行驗證,請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。
Java
Java
在試用這個範例之前,請先按照「使用用戶端程式庫的 Knowledge Catalog 快速入門導覽課程」中的 Java 設定說明操作。詳情請參閱 Knowledge Catalog Java API 參考文件。
如要向 Knowledge Catalog 進行驗證,請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。
Python
Python
在試用這個範例之前,請先按照「使用用戶端程式庫的 Knowledge Catalog 快速入門導覽課程」中的 Python 設定說明操作。詳情請參閱 Knowledge Catalog Python API 參考文件。
如要向 Knowledge Catalog 進行驗證,請設定應用程式預設憑證。詳情請參閱「為本機開發環境設定驗證機制」。
gcloud
如要查看目前的歷程設定,請使用 gcloud datalineage config describe 指令。您可以擷取專案、資料夾或組織的設定。
以下範例說明如何取得目前專案的設定:
gcloud datalineage config describe
舉例來說,如要取得特定專案的設定,請使用 --project 旗標:
gcloud datalineage config describe --project=PROJECT_ID
更改下列內容:
PROJECT_ID:要查看設定的專案 ID。
如要查看資料夾或組織的服務目前譜系擷取設定,請將 --project= 替換為下列其中一項:PROJECT_ID
--folder=:如要查看資料夾的資料擷取設定,請使用這個指令。FOLDER_ID--organization=:查看組織的資料擷取設定。ORGANIZATION_ID
REST
如要查看目前的歷程設定,請使用 projects.locations.config.get 方法。您可以擷取專案、資料夾或組織的設定。
以下範例說明如何取得專案設定:
使用任何要求資料之前,請先修改下列項目的值:
CLIENT_PROJECT_ID:用於計費或配額的用戶端專案 ID。PROJECT_ID:要查看設定的專案 ID。
HTTP 方法和網址:
GET https://datalineage.googleapis.com/v1/projects/PROJECT_ID/locations/global/config
請展開以下其中一個選項,以傳送要求:
指令會傳回下列其中一項輸出內容:
- 如未提供任何歷程擷取設定,輸出內容會包含空白的
ingestion物件:{ "name": "projects/123456789012/locations/global/config", "ingestion": {} }
這表示服務會使用預設歷程擷取設定。 在本範例中,Managed Service for Apache Spark 的歷程擷取設定為
enabled。 - 如果明確啟用歷程擷取功能,會看到下列輸出內容:
{ "name": "projects/123456789012/locations/global/config", "ingestion": { "rules": [ { "integrationSelector": { "integration": "DATAPROC" }, "lineageEnablement": { "enabled": true } } ] }, "etag": "1a2b3c4d5e" }
- 如果譜系擷取功能已停用,您會看到下列輸出內容:
{ "name": "projects/123456789012/locations/global/config", "ingestion": { "rules": [ { "integrationSelector": { "integration": "DATAPROC" }, "lineageEnablement": { "enabled": false } } ] }, "etag": "1a2b3c4d5e" }
如要取得資料夾或組織的設定,請將 projects/ 替換為 PROJECT_IDfolders/ 或 FOLDER_IDorganizations/。ORGANIZATION_ID
回應中的 etag 欄位是伺服器根據設定目前的值產生的核對和。使用 patch 方法更新設定時,您可以在要求主體中加入最近 get 要求傳回的 etag 值。如果您提供 etag,Knowledge Catalog 會使用該值,確認設定自上次讀取要求後未變更。如果不相符,更新要求就會失敗。這可避免您在讀取-修改-寫入情境中,不小心覆寫其他使用者所做的設定。如果您未在 patch 要求中提供 etag,Knowledge Catalog 會無條件覆寫設定。
停用服務的歷程擷取功能
如要控管成本、強制執行資料治理政策,或排除開發專案和其他不適用歷程追蹤的工作負載,請停用服務的歷程擷取功能。
Java
package com.google.cloud.datacatalog.lineage.configmanagement.v1.samples;
import com.google.api.gax.rpc.NotFoundException;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config.Ingestion;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config.Ingestion.IngestionRule;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config.Ingestion.IngestionRule.IntegrationSelector;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config.Ingestion.IngestionRule.IntegrationSelector.Integration;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config.Ingestion.IngestionRule.LineageEnablement;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.ConfigManagementServiceClient;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.ConfigName;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.GetConfigRequest;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.UpdateConfigRequest;
public class DisableLineageIngestion {
public static void main(String[] args) throws Exception {
// TODO(developer): Replace these variables before running the sample.
String projectId = "your-project-id";
String location = "global";
disableLineageIngestion(projectId, location);
}
// Disables lineage ingestion for a specific service
// (Managed Service for Apache Spark).
public static void disableLineageIngestion(String projectId, String location) throws Exception {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests.
try (ConfigManagementServiceClient client = ConfigManagementServiceClient.create()) {
// Format the resource name.
String name = ConfigName.ofProjectLocationName(projectId, location).toString();
Config.Builder configBuilder = Config.newBuilder().setName(name);
// It is a best practice to read the existing config to preserve other rules
// and use the etag for optimistic concurrency control.
try {
GetConfigRequest getRequest = GetConfigRequest.newBuilder().setName(name).build();
Config existingConfig = client.getConfig(getRequest);
configBuilder.mergeFrom(existingConfig);
} catch (NotFoundException e) {
// If config doesn't exist, we will proceed by creating a new one.
}
// Create an integration selector for the service you want to disable.
IntegrationSelector selector =
IntegrationSelector.newBuilder().setIntegration(Integration.DATAPROC).build();
// Set lineage enablement to false to disable tracking.
LineageEnablement enablement = LineageEnablement.newBuilder().setEnabled(false).build();
// Build the ingestion rule.
IngestionRule disableRule =
IngestionRule.newBuilder()
.setIntegrationSelector(selector)
.setLineageEnablement(enablement)
.build();
// Preserve existing rules except for the one we are modifying, then add the new rule.
// We clear the ingestion block out of the configBuilder entirely to reconstruct it.
Ingestion.Builder ingestionBuilder = Ingestion.newBuilder();
if (configBuilder.hasIngestion()) {
for (IngestionRule rule : configBuilder.getIngestion().getRulesList()) {
// Keep all existing rules EXCEPT the one targeting DATAPROC
if (rule.getIntegrationSelector().getIntegration() != Integration.DATAPROC) {
ingestionBuilder.addRules(rule);
}
}
}
ingestionBuilder.addRules(disableRule);
// Update the config builder with the reconstructed ingestion settings.
configBuilder.setIngestion(ingestionBuilder.build());
// Build the update request.
UpdateConfigRequest request = UpdateConfigRequest.newBuilder()
.setConfig(configBuilder.build())
.build();
// Update the config.
Config response = client.updateConfig(request);
System.out.printf("Successfully updated config: %s\n", response.getName());
}
}
}
Python
from google.api_core.exceptions import NotFound
from google.cloud.datacatalog.lineage import configmanagement_v1
def disable_lineage_ingestion(project_id: str, location: str = "global") -> configmanagement_v1.Config:
"""Disables lineage ingestion for a specific service.
Args:
project_id: The ID of your Google Cloud project.
location: The region location, usually 'global'.
Returns:
The updated Configuration object.
"""
# Initialize client that will be used to send requests.
client = configmanagement_v1.ConfigManagementServiceClient()
# The config name format
name = f"projects/{project_id}/locations/{location}/config"
try:
# Retrieve the existing config to preserve other configurations and
# obtain the latest etag for optimistic concurrency control.
config = client.get_config(name=name)
# Filter out existing rules for the integration we are updating
new_rules = [
rule for rule in config.ingestion.rules
if rule.integration_selector.integration != configmanagement_v1.Config.Ingestion.IngestionRule.IntegrationSelector.Integration.DATAPROC
]
except NotFound:
# If the config does not exist, start fresh
config = configmanagement_v1.Config(name=name)
new_rules = []
# Define the integration to disable tracking for (e.g., DATAPROC).
integration_selector = configmanagement_v1.Config.Ingestion.IngestionRule.IntegrationSelector(
integration=configmanagement_v1.Config.Ingestion.IngestionRule.IntegrationSelector.Integration.DATAPROC
)
# Set lineage enablement to False to disable tracking.
lineage_enablement = configmanagement_v1.Config.Ingestion.IngestionRule.LineageEnablement(
enabled=False
)
# Create the ingestion rule.
disable_rule = configmanagement_v1.Config.Ingestion.IngestionRule(
integration_selector=integration_selector,
lineage_enablement=lineage_enablement,
)
# Append the new disabling rule and assign it back to the config ingestion rules
new_rules.append(disable_rule)
config.ingestion = configmanagement_v1.Config.Ingestion(rules=new_rules)
# Create the update request using the config (which includes the etag if it existed).
request = configmanagement_v1.UpdateConfigRequest(
config=config,
)
# Make the request to update the config
response = client.update_config(request=request)
print(f"Successfully updated config: {response.name}")
return response
gcloud
如要停用特定服務的歷程擷取功能,請使用 gcloud datalineage config update 指令,並搭配內嵌 JSON 字串或 JSON 檔案路徑,將特定 integration 的 lineageEnablement.enabled 設為 false。
下列範例說明如何使用內嵌 JSON 字串,停用專案中某項服務的歷程擷取功能:
gcloud datalineage config update --project=PROJECT_ID \
--config='{
"ingestion": {
"rules": [
{
"integrationSelector": {
"integration": "INTEGRATION"
},
"lineageEnablement": {
"enabled": false
}
}
]
},
"etag": "ETAG"
}'
更改下列內容:
PROJECT_ID:要更新設定的專案 ID。INTEGRATION:要設定的整合項目。例如:DATAPROC或BIGQUERY。ETAG:要求主體中最近get要求傳回的etag值,用於驗證設定自上次讀取要求後未變更。
如要使用 JSON 檔案更新設定,請執行下列指令:
gcloud datalineage config update --project=PROJECT_ID --config=CONFIG_FILE
更改下列內容:
CONFIG_FILE:包含設定的 JSON 檔案路徑。
如要停用資料夾或組織的服務歷程擷取功能,請將 --project= 替換為下列其中一項:PROJECT_ID
--folder=:如要更新資料夾的資料擷取設定,請使用這個指令。FOLDER_ID--organization=:如要更新組織的資料擷取設定,請使用這個選項。ORGANIZATION_ID
REST
如要停用特定服務的歷程擷取功能,請使用 projects.locations.config.patch 方法,並設定擷取規則,將特定 integration 的 lineageEnablement.enabled 設為 false。
在讀取-修改-寫入情境中,為避免意外覆寫其他使用者所做的設定,您可以在要求主體中加入 etag 欄位。詳情請參閱「取得目前設定」。
使用任何要求資料之前,請先修改下列項目的值:
CLIENT_PROJECT_ID:用於計費或配額的用戶端專案 ID。PROJECT_ID:要更新設定的專案 ID。ETAG:最近get要求傳回的etag值。INTEGRATION:您設定的integration。例如:DATAPROC。
HTTP 方法和網址:
PATCH https://datalineage.googleapis.com/v1/projects/PROJECT_ID/locations/global/config
JSON 要求主體:
{
"ingestion": {
"rules": [
{
"integrationSelector": {
"integration": "INTEGRATION"
},
"lineageEnablement": {
"enabled": false
}
}
]
},
"etag": "ETAG"
}
請展開以下其中一個選項,以傳送要求:
您應該會收到如下的 JSON 回覆:
{
"name": "projects/PROJECT_ID/locations/global/config",
"ingestion": {
"rules": [
{
"integrationSelector": {
"integration": "INTEGRATION"
},
"lineageEnablement": {
"enabled": false
}
}
]
},
"etag": "1a2b3c4d5e"
}
如要停用資料夾或組織的歷程擷取功能,請將 projects/ 替換為 PROJECT_IDfolders/ 或 FOLDER_IDorganizations/。ORGANIZATION_ID
啟用服務的歷程擷取功能
如要在停用後繼續追蹤,或啟用預設停用的整合功能,請為服務啟用歷程擷取功能。
Java
package com.google.cloud.datacatalog.lineage.configmanagement.v1.samples;
import com.google.api.gax.rpc.NotFoundException;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config.Ingestion;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config.Ingestion.IngestionRule;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config.Ingestion.IngestionRule.IntegrationSelector;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config.Ingestion.IngestionRule.IntegrationSelector.Integration;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.Config.Ingestion.IngestionRule.LineageEnablement;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.ConfigManagementServiceClient;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.ConfigName;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.GetConfigRequest;
import com.google.cloud.datacatalog.lineage.configmanagement.v1.UpdateConfigRequest;
public class EnableLineageIngestion {
public static void main(String[] args) throws Exception {
// TODO(developer): Replace these variables before running the sample.
String projectId = "your-project-id";
String location = "global";
enableLineageIngestion(projectId, location);
}
// Enables lineage ingestion for a specific service
// (Managed Service for Apache Spark).
public static void enableLineageIngestion(String projectId, String location) throws Exception {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests.
try (ConfigManagementServiceClient client = ConfigManagementServiceClient.create()) {
// Format the resource name.
String name = ConfigName.ofProjectLocationName(projectId, location).toString();
Config.Builder configBuilder = Config.newBuilder().setName(name);
// It is a best practice to read the existing config to preserve other rules
// and use the etag for optimistic concurrency control.
try {
GetConfigRequest getRequest = GetConfigRequest.newBuilder().setName(name).build();
Config existingConfig = client.getConfig(getRequest);
configBuilder.mergeFrom(existingConfig);
} catch (NotFoundException e) {
// If config doesn't exist, we will proceed by creating a new one.
}
// Create an integration selector for the service you want to enable (e.g., DATAPROC).
IntegrationSelector selector =
IntegrationSelector.newBuilder().setIntegration(Integration.DATAPROC).build();
// Set lineage enablement to true to enable tracking.
LineageEnablement enablement = LineageEnablement.newBuilder().setEnabled(true).build();
// Build the ingestion rule.
IngestionRule enableRule =
IngestionRule.newBuilder()
.setIntegrationSelector(selector)
.setLineageEnablement(enablement)
.build();
// Preserve existing rules except for the one we are modifying, then add the new rule.
// We clear the ingestion block out of the configBuilder entirely to reconstruct it.
Ingestion.Builder ingestionBuilder = Ingestion.newBuilder();
if (configBuilder.hasIngestion()) {
for (IngestionRule rule : configBuilder.getIngestion().getRulesList()) {
// Keep all existing rules EXCEPT the one targeting DATAPROC
if (rule.getIntegrationSelector().getIntegration() != Integration.DATAPROC) {
ingestionBuilder.addRules(rule);
}
}
}
ingestionBuilder.addRules(enableRule);
// Update the config builder with the reconstructed ingestion settings.
configBuilder.setIngestion(ingestionBuilder.build());
// Build the update request.
UpdateConfigRequest request = UpdateConfigRequest.newBuilder()
.setConfig(configBuilder.build())
.build();
// Update the config.
Config response = client.updateConfig(request);
System.out.printf("Successfully updated config: %s\n", response.getName());
}
}
}
Python
from google.api_core.exceptions import NotFound
from google.cloud.datacatalog.lineage import configmanagement_v1
def enable_lineage_ingestion(project_id: str, location: str = "global") -> configmanagement_v1.Config:
"""Enables lineage ingestion for a specific service like Dataproc
(Managed Service for Apache Spark).
Args:
project_id: The ID of your Google Cloud project.
location: The region location, usually 'global'.
Returns:
The updated Configuration object.
"""
# Initialize client that will be used to send requests.
client = configmanagement_v1.ConfigManagementServiceClient()
# The config name format
name = f"projects/{project_id}/locations/{location}/config"
try:
# Retrieve the existing config to preserve other configurations and
# obtain the latest etag for optimistic concurrency control.
config = client.get_config(name=name)
# Filter out existing rules for the integration we are updating
new_rules = [
rule for rule in config.ingestion.rules
if rule.integration_selector.integration != configmanagement_v1.Config.Ingestion.IngestionRule.IntegrationSelector.Integration.DATAPROC
]
except NotFound:
# If the config does not exist, start fresh
config = configmanagement_v1.Config(name=name)
new_rules = []
# Define the integration to enable tracking for (e.g., DATAPROC).
integration_selector = configmanagement_v1.Config.Ingestion.IngestionRule.IntegrationSelector(
integration=configmanagement_v1.Config.Ingestion.IngestionRule.IntegrationSelector.Integration.DATAPROC
)
# Set lineage enablement to True to enable tracking.
lineage_enablement = configmanagement_v1.Config.Ingestion.IngestionRule.LineageEnablement(
enabled=True
)
# Create the ingestion rule.
enable_rule = configmanagement_v1.Config.Ingestion.IngestionRule(
integration_selector=integration_selector,
lineage_enablement=lineage_enablement,
)
# Append the new enabling rule and assign it back to the config ingestion rules
new_rules.append(enable_rule)
config.ingestion = configmanagement_v1.Config.Ingestion(rules=new_rules)
# Create the update request using the config (which includes the etag if it existed).
request = configmanagement_v1.UpdateConfigRequest(
config=config,
)
# Make the request to update the config
response = client.update_config(request=request)
print(f"Successfully updated config: {response.name}")
return response
gcloud
如要啟用特定服務的歷程擷取功能,請使用 gcloud datalineage config update 指令,並搭配內嵌 JSON 字串或 JSON 檔案路徑,將特定 integration 的 lineageEnablement.enabled 設為 true。目前整合的服務包括 Managed Service for Apache Spark、BigQuery 和 Managed Airflow。
下列範例說明如何使用內嵌 JSON 字串,為專案啟用服務歷程擷取功能:
gcloud datalineage config update --project=PROJECT_ID \
--config='{
"ingestion": {
"rules": [
{
"integrationSelector": {
"integration": "INTEGRATION"
},
"lineageEnablement": {
"enabled": true
}
}
]
},
"etag": "ETAG"
}'
更改下列內容:
PROJECT_ID:要更新設定的專案 ID。INTEGRATION:您要設定的整合項目 (例如DATAPROC或BIGQUERY)。ETAG:要求主體中最近get要求傳回的etag值,用於驗證設定自上次讀取要求後未變更。
如要使用 JSON 檔案更新設定,請執行下列指令:
gcloud datalineage config update --project=PROJECT_ID --config=CONFIG_FILE
更改下列內容:
CONFIG_FILE:包含設定的 JSON 檔案路徑。
如要為資料夾或組織啟用服務的歷程擷取功能,請將 --project= 替換為下列其中一項:PROJECT_ID
--folder=:如要更新資料夾的資料擷取設定,請使用這個選項。FOLDER_ID--organization=:如要更新組織的資料擷取設定,請使用這個選項。ORGANIZATION_ID
REST
如要為特定服務啟用歷程擷取功能,請使用 projects.locations.config.patch 方法,並搭配擷取規則,將特定 integration 的 lineageEnablement.enabled 設為 true。
在讀取-修改-寫入情境中,為避免意外覆寫其他使用者所做的設定,您可以在要求主體中加入 etag 欄位。詳情請參閱「取得目前設定」。
使用任何要求資料之前,請先修改下列項目的值:
CLIENT_PROJECT_ID:用於計費或配額的用戶端專案 ID。PROJECT_ID:要更新設定的專案 ID。ETAG:最近get要求傳回的etag值。INTEGRATION:您設定的integration。例如:DATAPROC。
HTTP 方法和網址:
PATCH https://datalineage.googleapis.com/v1/projects/PROJECT_ID/locations/global/config
JSON 要求主體:
{
"ingestion": {
"rules": [
{
"integrationSelector": {
"integration": "INTEGRATION"
},
"lineageEnablement": {
"enabled": true
}
}
]
},
"etag": "ETAG"
}
請展開以下其中一個選項,以傳送要求:
您應該會收到如下的 JSON 回覆:
{
"name": "projects/PROJECT_ID/locations/global/config",
"ingestion": {
"rules": [
{
"integrationSelector": {
"integration": "INTEGRATION"
},
"lineageEnablement": {
"enabled": true
}
}
]
},
"etag": "1a2b3c4d5e"
}
如要為資料夾或組織啟用服務的歷程擷取功能,請將 projects/ 替換為 PROJECT_IDfolders/ 或 FOLDER_IDorganizations/。ORGANIZATION_ID
為多項服務設定歷程擷取作業
如要同時為多項整合作業設定歷程擷取功能,請使用 projects.locations.config.patch、folders.locations.config.patch 或 organizations.locations.config.patch 方法。您可以在要求主體中定義多項規則,在專案、資料夾或組織層級更新設定。詳情請參閱「多服務整合的資料擷取設定運作方式」。
組織層級
設定組織,啟用 Managed Service for Apache Spark 的歷程擷取功能:
curl -X PATCH \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-H "X-Goog-User-Project: CLIENT_PROJECT_ID" \
"https://datalineage.googleapis.com/v1/organizations/ORGANIZATION_ID/locations/global/config" \
-d '{
"ingestion": {
"rules": [
{
"integrationSelector": {
"integration": "DATAPROC"
},
"lineageEnablement": {
"enabled": true
}
}
]
},
"etag": "ORGANIZATION_CONFIG_ETAG"
}'
更改下列內容:
ORGANIZATION_ID:要更新設定的組織 ID。CLIENT_PROJECT_ID:用於計費或配額的用戶端專案 ID。ORGANIZATION_CONFIG_ETAG:最近get要求傳回的組織設定etag值。
資料夾層級
設定資料夾,啟用 BigQuery 的歷程擷取功能:
curl -X PATCH \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-H "X-Goog-User-Project: CLIENT_PROJECT_ID" \
"https://datalineage.googleapis.com/v1/folders/FOLDER_ID/locations/global/config" \
-d '{
"ingestion": {
"rules": [
{
"integrationSelector": {
"integration": "BIGQUERY"
},
"lineageEnablement": {
"enabled": true
}
}
]
},
"etag": "FOLDER_CONFIG_ETAG"
}'
更改下列內容:
FOLDER_ID:要更新設定的資料夾 ID。CLIENT_PROJECT_ID:用於計費或配額的用戶端專案 ID。FOLDER_CONFIG_ETAG:最近get要求傳回的資料夾設定etag值。
專案層級
設定專案,同時停用 BigQuery 的歷程擷取功能,並啟用 Managed Service for Apache Airflow 的歷程擷取功能:
curl -X PATCH \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-H "X-Goog-User-Project: CLIENT_PROJECT_ID" \
"https://datalineage.googleapis.com/v1/projects/PROJECT_ID/locations/global/config" \
-d '{
"ingestion": {
"rules": [
{
"integrationSelector": {
"integration": "BIGQUERY"
},
"lineageEnablement": {
"enabled": false
}
},
{
"integrationSelector": {
"integration": "MANAGED_AIRFLOW"
},
"lineageEnablement": {
"enabled": true
}
}
]
},
"etag": "PROJECT_CONFIG_ETAG"
}'
更改下列內容:
PROJECT_ID:要更新設定的專案 ID。CLIENT_PROJECT_ID:用於計費或配額的用戶端專案 ID。PROJECT_CONFIG_ETAG:最近一次get要求傳回的專案設定etag值。
後續步驟
- 瞭解如何控管資料歷程擷取作業。
- 瞭解如何查看歷程資訊。