This document provides instructions on how to search for data assets in Knowledge Catalog.
Before you begin
Before you perform search, ensure that you are granted the required roles and have enabled the necessary API.
Required roles
To get the permissions that you need to search for entries and access search results in Knowledge Catalog, ask your administrator to grant you the following IAM roles:
-
Search for entries:
- Dataplex Catalog Admin (
roles/dataplex.catalogAdmin) on the project used for search - Dataplex Catalog Editor (
roles/dataplex.catalogEditor) on the project used for search - Dataplex Catalog Viewer (
roles/dataplex.catalogViewer) on the project used for search
- Dataplex Catalog Admin (
-
Search for metadata for BigQuery datasets and tables:
BigQuery Metadata Viewer (
roles/bigquery.metadataViewer) on the dataset or table -
Search for custom entries:
Dataplex Catalog Viewer (
roles/dataplex.catalogViewer) on the project used for search
For more information about granting roles, see Manage access to projects, folders, and organizations.
You might also be able to get the required permissions through custom roles or other predefined roles.
Asset-level permissions
Search results are scoped to your access level, independently of the selected project. To find an asset in Knowledge Catalog, you must have the appropriate read permissions for that resource in its underlying source system.
For example, finding a BigQuery table requires the
roles/bigquery.metadataViewer role, while finding a Cloud SQL instance
requires its equivalent
Cloud SQL permissions. For more
details, see Search scope.
Enable the API
Enable the Dataplex API.
Roles required to enable APIs
To enable APIs, you need the serviceusage.services.enable permission. If you
created the project, then you likely already have this permission through the
Owner role (roles/owner). Otherwise, you can get this permission through the
Service Usage Admin role (roles/serviceusage.serviceUsageAdmin).
Learn how to grant roles.
Search for resources
Console
To search for resources, follow these steps:
In the Google Cloud console, go to the Knowledge Catalog Search page.
If you see the Try natural language search button prompt, click it. By default, natural language search is selected.
In the Find resources across projects with natural language field, enter your query and then click Enter.
To refine your search, click Filters.
- Filters in multiple sections are evaluated with the
ANDlogical operator. - Multiple filters within a single section are evaluated with the
ORlogical operator.
The following filters are available:
- Scope: search across the organization (default), the current project, or only for starred resources. For more information, see Search scope.
- Systems: the Google Cloud service that the resource belongs to, such as BigQuery. The Knowledge Catalog system contains entry groups.
- Projects: the projects to search in.
- Type: the resource type, such as BigQuery connection, Cloud Storage bucket, or database. Depending on the resource type, you can also filter by subtype, such as the connection type or SQL dialect.
- Select locations: the locations to search in.
- Select datasets: the search results are limited to BigQuery resources that belong to the selected BigQuery datasets. In the Type to filter field, enter the name of the dataset.
- Aspect types: the Knowledge Catalog aspect types that are associated with the resource that you're searching for. To filter by aspect values, click Filter on aspect type values, and then select the values.
- Filters in multiple sections are evaluated with the
To view more information about the searched resource, click the resource name in the search results to open the entry details page.
Google Cloud CLI
To search for resources, use the
gcloud dataplex entries search command:
gcloud dataplex entries search 'foo' \ --project=PROJECT_ID \ --semantic-search
Replace PROJECT_ID with the ID of the Google Cloud project.
C#
Before trying this sample, follow the C# setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog C# API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
C#
Go
Before trying this sample, follow the Go setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog Go API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Go
Java
Before trying this sample, follow the Java setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog Java API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Java
Node.js
Before trying this sample, follow the Node.js setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog Node.js API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Node.js
PHP
Before trying this sample, follow the PHP setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog PHP API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
PHP
Python
Before trying this sample, follow the Python setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog Python API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Python
Ruby
Before trying this sample, follow the Ruby setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog Ruby API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Ruby
REST
To search for resources, use the
searchEntries method with the SemanticSearch parameter set to true.
POST https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION:searchEntries?query=foo&semanticSearch=true
Replace the following:
- PROJECT_ID: the ID of your Google Cloud project
- LOCATION: the region where the project exists (for example,
us-central1)
View details of an asset returned by search
Console
Use Knowledge Catalog search to view the details of an asset.
Search for an asset in Knowledge Catalog.
In the search results, click the asset for which you want to view the details.
The entry details page opens. The page includes the following sections:
- Entry details: includes information such as the entry type, system, platform, fully qualified name, creation time, last modification time, description, and stewards.
- Overview: an overview of the entry, if available.
- Aspects: the required and optional aspects defined for the entry. For more information, see Categories of aspects.
gcloud
gcloud dataplex entries lookup command:
gcloud dataplex entries lookup ENTRY_ID \ --entry-group=ENTRY_GROUP_ID \ --location=LOCATION \ --project=PROJECT_ID
Replace the following:
ENTRY_ID: the ID of the entryENTRY_GROUP_ID: the ID of the entry groupLOCATION: the region where the project existsPROJECT_ID: the ID of the Google Cloud project
C#
Before trying this sample, follow the C# setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog C# API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
C#
Go
Before trying this sample, follow the Go setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog Go API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Go
Java
Before trying this sample, follow the Java setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog Java API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Java
Node.js
Before trying this sample, follow the Node.js setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog Node.js API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Node.js
PHP
Before trying this sample, follow the PHP setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog PHP API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
PHP
Python
Before trying this sample, follow the Python setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog Python API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Python
Ruby
Before trying this sample, follow the Ruby setup instructions in the
Knowledge Catalog quickstart using
client libraries.
For more information, see the
Knowledge Catalog Ruby API
reference documentation.
To authenticate to Knowledge Catalog, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Ruby
REST
To view the details of an asset, use the lookupEntry method.
Limitations
Search has the following limitations:
- Public resources are outside the scope of natural language search.
- Aspects attached to entry links are outside the scope of natural language search.
What's next
- Understand search syntax for Knowledge Catalog.
- Learn more about metadata management in Knowledge Catalog.
- Learn how to enrich entries and entry links with metadata using aspects.
- Learn how to manage entries and ingest custom sources.
- Try Knowledge Catalog use cases.
- Leverage answering complex natural language queries with the Knowledge Catalog discovery agent.