This document describes the syntax for Knowledge Catalog search.
Knowledge Catalog lets you discover, centrally catalog, manage, and understand your organization's data. To efficiently find specific data assets within your data catalog, you can use powerful search queries. The syntax for search queries includes:
- Simple search: How to find data assets using a single search term.
- Qualified predicates: How to refine your search using specific metadata fields like name, location, or description.
- Aspect search: How to search for entries based on their attached business and technical metadata.
- Logical operators: How to combine multiple search criteria using AND and OR to create complex queries. By understanding this syntax, you can quickly locate the data you need.
Before you begin
Understand Knowledge Catalog entry groups, entries, entry links, aspects, entry types, entry link types, and aspect types. See About metadata management in Knowledge Catalog.
Knowledge Catalog offers search that leverages AI to understand natural language queries and semantic meaning. It lets you find resources using everyday language, eliminating the need for complex syntax, while also supporting keyword matching and filters.
Use free-text search
You can find assets by entering a term or phrase without any specific syntax. Knowledge Catalog performs a broad search by matching your query against several metadata fields, including the following:
- Name, display name, or description of a resource
- Type of a resource
- Project ID
- Overview description
- Column name (or nested column name) in the schema of a resource
- Column description
- Fully qualified name
- Contacts
- Aspects
Use query syntax
For more precise searches, you can construct a query using specific syntax, including qualifiers, logical operators, and aspect searches.
Qualified predicates
You can qualify a predicate by prefixing it with a key that restricts the matching to a specific piece of metadata:
- An equal sign (
=) restricts the search to an exact match. - A colon (
:) after the key matches the predicate to either a substring or a token within the value in the search results.
Tokenization splits the stream of text into a series of tokens, with each token usually corresponding to a single word.
For example:
name:fooselects resources with names that contain thefoosubstring, such asfoo1andbarfoo.description:fooselects resources with thefootoken in the description, such asbarandfoo.location=foomatches resources in a specified location withfooas the location name.
The behavior of these qualifiers can vary slightly between search modes, as detailed in the following sections.
The predicate keys type, system, location, and description, and aspect
search (excluding has) support only the exact match (=) qualifier, not the
substring qualifier (:). For example, type=foo.
Knowledge Catalog search supports the following qualifiers:
| Qualifier | Description |
|---|---|
name:x |
Matches x as a substring of the resource ID or resource
display name. |
displayname:x |
Match x as a substring of the resource display name. |
column:x |
Matches x as a substring of the column name (or nested
column name) in the schema of the resource. |
description:x |
Matches x as a token in the resource description. |
labels:bar |
Matches BigQuery resources that have a label (with some
value) and the label key has bar as a substring. |
labels=bar |
Matches BigQuery resources that have a label (with
some value) and the label key equals bar as a string. |
labels.bar:x |
Matches x as a substring in the value of a label with
key bar attached to a BigQuery resource. |
labels.foo=bar |
Matches BigQuery resources where the key equals
foo and the key value equals bar. |
type=TYPE |
Matches resources of a specific entry type or its type alias. |
projectid:bar |
Matches resources within Google Cloud projects that match
bar as a substring in the ID. |
parent:x |
Matches x as a substring of the hierarchical path of a
resource. |
system=SYSTEM |
Matches resources from a specified system. |
location=LOCATION |
Matches resources in a specified location with an exact name.
For example, BigQuery Omni assets support this qualifier by using the
BigQuery Omni location name.
For example,
|
createtime |
Finds resources that were created within, before, or after a given date or time. For example:
Timestamp format: All timestamps must be in GMT; time zones are not supported. Partial
timestamps, hyphen ( For example:
|
updatetime |
Finds resources that were updated within, before, or after a given date or time. For example:
Timestamp format: All timestamps must be in GMT; time zones are not supported. Partial
timestamps, hyphen ( For example:
|
Aspect search
You can use query syntax to search for entries based on their attached aspects.
The substring match tries to match against a limited number of aspects. If you can't find the entry by using a fragment of the path, then use the full path to narrow the search and increase recall.
| Qualifier | Description |
|---|---|
aspect:xor has:x |
Matches x as a substring of the full path to the aspect
type of an aspect that is attached to the entry, in the format
projectid.location.ASPECT_TYPE_ID |
aspect=xor has=x |
Matches x as the full path to the aspect
type of an aspect that is attached to the entry, in the format
projectid.location.ASPECT_TYPE_ID |
x |
Searches for aspect field values. Matches
Only top-level fields of the aspect are searchable. |
Logical operators
A query can consist of several predicates with logical operators.
- If you don't specify an operator, logical
ANDis implied. For example,foo barreturns resources that match both predicatefooand predicatebar. - Logical
ANDand logicalORare supported. For example,foo OR bar. - You can negate a predicate with a
-(hyphen) orNOTprefix. For example,-name:fooreturns resources with names that don't match the predicatefoo.
Abbreviated syntax
An abbreviated search syntax is also available, using | (vertical bar) for
OR operators and , (comma) for AND operators.
For example, to search for entries inside one of many projects using the OR
operator, you can use the following abbreviated syntax:
projectid:(id1|id2|id3|id4)
The same search without using abbreviated syntax looks like the following:
projectid:id1 OR projectid:id2 OR projectid:id3 OR projectid:id4
To search for entries with matching column names, use the following:
- AND:
column:(name1,name2,name3) - OR:
column:(name1|name2|name3)
This abbreviated syntax works for the
qualified predicates except for label in keyword
search.
Keyword-only search
Knowledge Catalog provides keyword-only search mode for backward compatibility, with the standard search mode supporting both semantic and keyword matching. We recommend using the standard search mode, unless keyword-only search is required for backward compatibility.
The following table summarizes the key differences in qualifiers that you must consider when using keyword-only search.
| Qualifier | Key differences |
|---|---|
name:x |
Matches x as a substring of the resource ID only. |
label qualifiers |
Uses the
|
orgid=number |
Matches resources within a Google Cloud organization with
the exact ID value of number, supported only in
keyword-only search. |
fully_qualified_name |
Supports fully_qualified_name:x (substring match)
and fully_qualified_name=x (exact match) (supported
only in keyword-only search). |
createtime and updatetime |
Supports only exact date (:) and basic comparison
(<, >) operators. Doesn't support
relative time filters (such as -30d or -1d)
or comparison operators such as <=,
>=, =, =>,
=<. |
| Exact match restriction | The predicate keys type, system,
location, and orgid support only exact match
(=), not substring match (:). |
Aspect search for keyword-only search
The following table summarizes the key differences in aspect search that you must consider when using keyword-only search.
| Aspect search element | Key differences |
|---|---|
| Qualifier prefix | Supports aspect: and aspect=. Doesn't
support has: or has=. |
| Aspect field value search syntax | Requires the aspect: prefix when searching for aspect
field values (format: aspect:x OPERATOR value, such as
aspect:employee-info.is-enrolled=true or
aspect:employee=true). |
| String field operators | String fields support both exact match (=) and substring
match (:). |
| System aspect type shorthands | Doesn't support system aspect type shorthands, such as
ASPECT_TYPE_ID.FIELD_NAME or
dataplex-types...). |
What's next
- Learn how to search for resources in Knowledge Catalog
- Learn more about metadata management in Knowledge Catalog.
- Learn how to enrich entries and entry links with metadata by using aspects.
- Learn how to manage entries and ingest custom sources.