Use data domains

As a data domain user, you can use data domains to narrow search results to specific business contexts, search for BigQuery datasets, tables, and data products, and discover metadata within specific domain boundaries.

Before you begin

Before you perform a search by domain, ensure that you have the required roles and APIs enabled.

Enable APIs

Enable the Dataplex API, if it is not already enabled.

Roles required to enable APIs

To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

Enable the API

Required roles

To get the permissions that you need to search and discover metadata using data domains, ask your administrator to grant you the following IAM roles on the project or data domain:

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Browse the domain hierarchy

For more information on how to browse the domain hierarchy, see View data domains and subdomains in a location.

Find resources within a data domain

To search for resources with a domain, follow these steps:

Console

  1. In the Google Cloud console, go to the Knowledge Catalog > Search page.

    Go to Search

  2. Click Try natural search to activate the natural search feature.

  3. Click Filters.

  4. In the Scope field, select Domain from the list.

  5. In the Select domain field, select the domain you want to narrow the results to. Add any additional filters if necessary.

Alternatively, you can use the Search within domain menu option to find resources in the domain:

  1. In the Google Cloud console, go to the Data domains page.

    Go to Data domains

  2. Select Search within domain from the menu. This action automatically opens a search window with the domain filter applied.

  3. Use the search field to find resources in the domain.

REST

To search entries scoped to a data domain, use the projects.locations.searchEntries method and provide the domain path in the contexts parameter.

Before using any of the request data, make the following replacements:

  • PROJECT_ID: the ID of your Google Cloud project.
  • LOCATION_ID: the region for the search inquiry (or global).
  • QUERY: the search query string.
  • DOMAIN_ID: the unique identifier of the data domain defining the search scope context.

HTTP method and URL: 

POST https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID:searchEntries

Request JSON body: 

{
  "query": "QUERY",
  "contexts": [ "projects/PROJECT_ID/locations/LOCATION_ID/dataDomains/DOMAIN_ID" ],
  "semantic_search": true,
  "page_size": 100
}

To send your request, expand one of these options:

You should receive a successful status code (2xx) and an empty response.

Retrieve context for data domains

You can use the lookupContext method to ground AI agents in the business context captured by your data domains. The lookupContext method supports data domains in the following ways:

  • Context for a data domain. You can pass the entry that represents a data domain in the resources list. The response contains the metadata of the data domain, such as its description and the parent data domains in the domain hierarchy.
  • Data domain membership of data assets. When you retrieve context for a data asset that is included in one or more data domains, the response contains a dataDomains field that lists the data domains the asset belongs to, starting with the most specific domain and ascending to the top-level domain.

To retrieve context for a data domain, send the following request:

curl --request POST \
  'https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION:lookupContext' \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H 'Content-Type: application/json' \
  -d '{
    "resources": [
      "projects/PROJECT_ID/locations/LOCATION/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION/dataDomains/DATA_DOMAIN_ID"
    ],
    "options": {
      "format": "yaml"
    }
  }'

Replace the following:

  • PROJECT_ID: the ID of the project that contains the data domain.
  • PROJECT_NUMBER: the number of the project that contains the data domain.
  • LOCATION: the location of the data domain.
  • DATA_DOMAIN_ID: the ID of the data domain.

For more information about the lookupContext method, see Retrieve context for data assets.

Limitations

When you search for resources outside the domain context, for example when using a global search, the domain's discoverability permissions don't apply. You only see resources that you have access to in the underlying system, such as BigQuery.

Suppose a BigQuery dataset is included in a data domain. You have the Dataplex Data Domain Entry Reader role for the domain, but you don't have direct permission to view the dataset in BigQuery:

  • Searching within the domain: You see the dataset in your search results because the domain permissions apply.
  • Searching outside the domain: You don't see the dataset in your search results. In a general search, domain permissions don't apply, and you don't have direct access in BigQuery.

What's next