Use Knowledge Catalog with BigQuery

Knowledge Catalog, (formerly known as Dataplex Universal Catalog), interacts with BigQuery as a central data governance and agentic access layer for BigQuery's metadata. For more information, see the Knowledge Catalog overview.

How do I use Knowledge Catalog with BigQuery?

Knowledge Catalog interacts with BigQuery in the following ways.

Automated metadata ingestion

Knowledge Catalog automatically discovers and indexes technical metadata from BigQuery assets. This includes the following:

Metadata representation and enrichment

  • Entries: each BigQuery table or asset is represented as an entry in the catalog, rather than the entire table; for example, project.dataset.table.
  • Data quality scorecard integration: Knowledge Catalog provides a data quality scorecard for each entry in the catalog. In addition to built-in Knowledge Catalog data scans, assertion results from Dataform are a source for this scorecard.

  • Automated metadata enrichment: for tables and views created with Dataform, you can define semantic metadata directly in your code. This metadata is automatically synchronized to Knowledge Catalog upon successful completion of a pipeline action. The following aspect types are supported:

    • Overview: documentation and summary text for the entry.
    • Generic aspects: semantic metadata tags for technical details, such as table system and type information.

    To check the status of a metadata update, follow the instructions in View past manual runs.

    After the metadata has been synchronized, you can search for and view the entry in Knowledge Catalog. For more information, see Search for resources.

  • Column-level metadata: individual columns or fields are represented as paths, allowing you to attach specific metadata, like PII markers or data quality scores, to individual fields within a BigQuery table rather than just the table itself.

  • Aspects: technical metadata is enriched with aspects, which add business context to data, such as ownership, data quality, and documentation.

  • Data products: you can package related BigQuery assets into data products—for example, ecommerce business data—which have shared access and governance constraints.

  • Semantic search: users can use natural language to search for BigQuery data, which is especially useful for data scientists and AI agents to find trusted data products using long or complex queries.
  • Name translation: for easier programmatic lookup, the system allows for translating BigQuery SQL names, or fully qualified names, into Knowledge Catalog entry names.

Agentic access and grounding

  • Agentic access: AI agents can discover and adaptively use Knowledge Catalog tools through a local or remote MCP server.
  • Context for AI Agents: Knowledge Catalog curates a context graph that links BigQuery datasets with business semantics, helping reduce AI hallucinations by ensuring models use enterprise-approved data.

Governance and compliance

  • Data Lineage: Knowledge Catalog automatically tracks how data flows and transforms into and out of BigQuery tables. This capability is critical for auditing sensitive information, like PII, across the data estate.
  • Access Control: metadata management is integrated with Identity and Access Management (IAM) and VPC Service Controls to ensure that discovery and access to BigQuery metadata follow organizational security policies.

What's next

Learn more about Knowledge Catalog: