Use data domains

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

As a data domain user, you can use data domains to narrow search results to specific business contexts, search for BigQuery datasets, tables, and data products, and discover metadata within specific domain boundaries.

Before you begin

Before you perform a search by domain, ensure that you have the required roles and APIs enabled.

Enable APIs

Enable the Dataplex API.

Roles required to enable APIs

To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

Enable the API

Required roles

To get the permissions that you need to search and discover metadata using data domains, ask your administrator to grant you the following IAM roles on the project or data domain:

• View metadata of domain entries for discovery: Dataplex Data Domain Entry Reader (roles/dataplex.dataDomainEntryReader)
• View domain configurations and bindings: Dataplex Data Domain Configuration Viewer (roles/dataplex.dataDomainViewer)

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Browse the domain hierarchy

For more information on how to browse the domain hierarchy, see View data domains and subdomains in a location.

Find resources within a data domain

To search for resources with a domain, follow these steps:

Console

1. In the Google Cloud console, go to the Knowledge Catalog > Search page.

   Go to Search

2. Click Try natural search to activate the natural search feature.

3. Click Filters.

4. In the Scope field, select Domain from the list.

5. In the Select domain field, select the domain you want to narrow the results to. Add any additional filters if necessary.

Alternatively, you can use the Search within domain menu option to find resources in the domain:

1. In the Google Cloud console, go to the Data domains page.

   Go to Data domains

2. Select Search within domain from the more_vert menu. This action automatically opens a search window with the domain filter applied.

3. Use the search field to find resources in the domain.

REST

To search entries scoped to a data domain, use the projects.locations.searchEntries method and provide the domain path in the contexts parameter.

Note: The location in the path for searchEntries can be global (for full fanout) or a specific region depending on your search backend configuration.

Before using any of the request data, make the following replacements:

• PROJECT_ID: the ID of your Google Cloud project.
• LOCATION_ID: the region for the search inquiry (or global).
• QUERY: the search query string.
• DOMAIN_ID: the unique identifier of the data domain defining the search scope context.

HTTP method and URL:

POST https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID:searchEntries

Request JSON body:

{
  "query": "QUERY",
  "contexts": [ "projects/PROJECT_ID/locations/LOCATION_ID/dataDomains/DOMAIN_ID" ],
  "semantic_search": true,
  "page_size": 100
}

To send your request, expand one of these options:

curl (Linux, macOS, or Cloud Shell)

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login , or by using Cloud Shell, which automatically logs you into the gcloud CLI . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID:searchEntries"

PowerShell (Windows)

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID:searchEntries" | Select-Object -Expand Content

You should receive a successful status code (2xx) and an empty response.

Limitations

When you search for resources outside the domain context, for example when using a global search, the domain's discoverability permissions don't apply. You only see resources that you have access to in the underlying system, such as BigQuery.

Suppose a BigQuery dataset is included in a data domain. You have the Dataplex Data Domain Entry Reader role for the domain, but you don't have direct permission to view the dataset in BigQuery:

• Searching within the domain: You see the dataset in your search results because the domain permissions apply.
• Searching outside the domain: You don't see the dataset in your search results. In a general search, domain permissions don't apply, and you don't have direct access in BigQuery.

What's next

• Learn more about data domains.
• Learn how to create data domains.
• Learn how to manage data domain resources.