Managed connectivity pipelines let you extract and import metadata at scale
from third-party data sources into Knowledge Catalog (formerly Dataplex Universal Catalog) by using
custom connectors and Workflows. You can run these pipelines on
demand or on a recurring schedule to keep your catalog synchronized and
discover, search, and govern external assets in one place.

## How managed connectivity works

The following diagram shows a managed connectivity pipeline.

![A managed connectivity pipeline.](https://docs.cloud.google.com/static/knowledge-catalog/images/managed-connectivity-diagram.png)

At a high level, here's how managed connectivity works:

1. You
   [build a custom connector](https://docs.cloud.google.com/knowledge-catalog/docs/develop-custom-connector.md.txt)
   to extract metadata from various third-party data sources, including MySQL,
   SQL Server, Oracle, Snowflake, and Databricks. Alternatively, you can use
   [community-contributed custom connectors](https://docs.cloud.google.com/knowledge-catalog/docs/managed-connectivity-overview.md.txt#community-contributed-connectors)
   for a wider range of sources.

   The connector must be an Artifact Registry image that can run on
   Managed Service for Apache Spark.
2. You [run the managed connectivity pipeline](https://docs.cloud.google.com/knowledge-catalog/docs/import-using-workflows-custom-source.md.txt)
   in Workflows, an orchestration platform.

3. The managed connectivity pipeline performs the following actions:

   1. Creates a target Knowledge Catalog entry group in your Google Cloud project based on your configuration, if the entry group doesn't exist.
   2. Runs the connector. The connector extracts the metadata from your data source and generates a metadata import file that can be imported into Knowledge Catalog.
   3. Monitors the progress of the metadata extraction.
   4. Runs a metadata import job to import the metadata into Knowledge Catalog.
   5. Monitors the progress of the metadata import job.

The managed connectivity pipeline uses Managed Service for Apache Spark to run the
connector, and Knowledge Catalog metadata import API methods
to run the metadata import job.

The metadata that you import consists of Knowledge Catalog
entries and their aspects. For more information about
Knowledge Catalog metadata, see
[About metadata management in Knowledge Catalog](https://docs.cloud.google.com/knowledge-catalog/docs/metadata-overview.md.txt).

## Community-contributed custom connectors

To import metadata from third-party sources, you can use custom connectors that
are contributed by the community. See each connector's README file for setup
instructions and more information about the connector.

> [!NOTE]
> **Note:** These connectors are not officially supported by Google.

| Data source | Repository |
|---|---|
| AWS Glue Data Catalog | [aws-glue-connector](https://github.com/GoogleCloudPlatform/cloud-dataplex/tree/main/managed-connectivity/community-contributed-connectors/aws-glue-connector) |
| Databricks Unity Catalog | [databricks-connector](https://github.com/GoogleCloudPlatform/cloud-dataplex/tree/main/managed-connectivity/community-contributed-connectors/databricks-connector) |
| MySQL | [mysql-connector](https://github.com/GoogleCloudPlatform/cloud-dataplex/tree/main/managed-connectivity/community-contributed-connectors/mysql-connector) |
| Oracle | [oracle-connector](https://github.com/GoogleCloudPlatform/cloud-dataplex/tree/main/managed-connectivity/community-contributed-connectors/oracle-connector) |
| PostgreSQL | [postgresql-connector](https://github.com/GoogleCloudPlatform/cloud-dataplex/tree/main/managed-connectivity/community-contributed-connectors/postgresql-connector) |
| Snowflake | [snowflake-connector](https://github.com/GoogleCloudPlatform/cloud-dataplex/tree/main/managed-connectivity/community-contributed-connectors/snowflake-connector) |
| Snowflake Horizon Catalog | [snowflake-horizon-connector](https://github.com/GoogleCloudPlatform/cloud-dataplex/tree/main/managed-connectivity/community-contributed-connectors/snowflake-horizon-connector) |
| SQL Server | [sql-server-connector](https://github.com/GoogleCloudPlatform/cloud-dataplex/tree/main/managed-connectivity/community-contributed-connectors/sql-server-connector) |

## What's next

- [Import metadata from a custom source using Workflows](https://docs.cloud.google.com/knowledge-catalog/docs/import-using-workflows-custom-source.md.txt)
- [Develop a custom connector for metadata import](https://docs.cloud.google.com/knowledge-catalog/docs/develop-custom-connector.md.txt)
- [Import metadata using a custom pipeline](https://docs.cloud.google.com/knowledge-catalog/docs/import-metadata.md.txt)