About search in Knowledge Catalog

Use Knowledge Catalog (formerly Dataplex Universal Catalog) search to discover resources across your organization using natural language queries with semantic matching, keywords, and structured predicates.

Use cases

The following list describes common search use cases, along with descriptions and example queries:

  • Targeted asset lookup: Find a specific asset by searching for keywords related to the asset name, columns, or description.

    Example queries

    • retail_transactions_2026
    • customer_id
    • column:customer_id

  • Broad data discovery: Identify relevant assets across your organization using open-ended natural language or keyword queries.

    Example queries

    • quarterly financial reports
    • ad campaign click through rates tables
    • server health metrics
    • audit logs system=bigquery

  • Scoped asset retrieval for workflows: Enumerate assets within a specific container, such as a project, or with specific properties, such as type or system. This approach is frequently used for programmatic and agentic workflows.

    Example queries

    • type=table projectid:banking-prod aspect:classification.tier=PII
    • system=spanner projectid:inventory-service (parent=marketing_analytics OR parent=finance_analytics)

How it works

Search automatically indexes all context for data assets maintained in Knowledge Catalog. This includes the following:

  • Metadata automatically ingested from Google Cloud data sources such as BigQuery and Cloud SQL
  • Context for resources you ingest through connectors and integrations
  • Additional context you create for resources, for example representing business context or describing data semantics. The context might be also captured in the form of aspects or linked business terms.

When processing your query, search applies a combination of semantic and keyword matching. The following table describes the query types you can use with Knowledge Catalog search along with descriptions and sample queries:

Query type Usage Examples
Single keyword For exact and substring matching across metadata elements, such as asset name, description, and schema. prd_fin_invoices_fct_v02
Partial keywords and tokenized fragments Matches substrings across metadata content, finding resources even with abbreviated terms, separated words, or naming variations. fin transactions 2026 (matches prd_fin_transactions_fy2026_raw)
Natural language queries Uses semantic matching without requiring exact name or column matches. customer churn prediction features
Structured predicates Combines free-text queries with explicit predicate filters for narrowing down search results. audit logs system=bigquery
Extended syntax For precise search, narrowed down to a specific scope. Extended syntax is most often used in agentic and programmatic use cases. For more information, see Search syntax. (system:bigquery OR system:spanner) AND column:credit_card_number aspect:classification.tier=PII -projectid:sandbox-project

You have the following options to access search in Knowledge Catalog:

Search scope

The search results in Knowledge Catalog respect permissions that you have over the corresponding resources in source systems.

For example, if you have BigQuery metadata read access to an object, that object appears in your Knowledge Catalog search results. If you have access to a BigQuery table but not to the dataset containing that table, the table still appears as expected in the Knowledge Catalog search.

By default, search is scoped to your organization. Results include only resources from the same organization as the project you're searching in.

The search results include only those resources that belong to the same VPC Service Controls perimeter as the project under which search is performed. When using the Google Cloud console, this is the project that is selected in the console.

To broaden the scope of your search results beyond the resources within your project's VPC Service Controls perimeter, use VPC Service Controls ingress and egress rules. These rules facilitate private and efficient data exchange across your organization. You can configure ingress and egress rules using the Google Cloud console or through JSON or YAML files. Refer to the following YAML example and consult the VPC Service Controls documentation to tailor the rule to your specific requirements.

egressPolicies:
  - egressFrom:
      identityType: ANY_USER_ACCOUNT
    egressTo:
      # Specify which resources should be present in the search results. In this example,
      # BigQuery.
      operations:
      - methodSelectors:
        - method: '*'
        serviceName: bigquery.googleapis.com
      # Specify project ids under which the search is performed.
      resources:
      - projects/SEARCH_PROJECT_ID
ingressPolicies:
  - ingressFrom:
      identityType: ANY_USER_ACCOUNT
      sources:
      - accessLevel: '*'
    ingressTo:
      # Specify which resources should be present in the search results. In this example,
      # BigQuery.
      operations:
      - methodSelectors:
        - method: '*'
        serviceName: bigquery.googleapis.com
      # Specify project ids to expose in search results.
      resources:
      - projects/INGRESS_PROJECT_ID

For more information about the Identity and Access Management roles that you need to use Knowledge Catalog search, see Knowledge Catalog IAM roles.

Isolate search results by environment using VPC Service Controls

To isolate Knowledge Catalog search results between environments like development, test, and production, configure separate VPC Service Controls perimeters for each environment. Assign both the projects that contain the data assets and the projects that are used for performing searches to the corresponding environment's perimeter. Searches that are performed from a project within a specific perimeter only returns results for assets that are located within that same perimeter.

Recall limitations in search

Knowledge Catalog search queries don't guarantee full recall, which means that the search might not return results that match your query. Additionally, returned (and not returned) results might vary if you repeat search queries.

To query all Knowledge Catalog metadata, you can export the metadata to Cloud Storage and then query it from BigQuery. For more information, see Export metadata.

What's next