Knowledge Catalog (formerly Dataplex Universal Catalog) lets you bulk import glossaries, categories,
terms, and entry links using the Dataplex API. You can use this process to
migrate metadata from other cataloging tools or perform bulk updates to your
existing glossaries.

## Before you begin

Before you start the import process, complete the following prerequisites:

### Create a Cloud Storage bucket

[Create a Cloud Storage bucket](https://docs.cloud.google.com/storage/docs/creating-buckets.md.txt) to
serve as a staging area where you upload JSON files for import.

### Required roles


To get the permissions that
you need to import glossaries and entry links using JSON files,

ask your administrator to grant you the
following IAM roles:

- [Dataplex Metadata Editor](https://docs.cloud.google.com/iam/docs/roles-permissions/dataplex.md.txt#dataplex.metadataEditor) (`roles/dataplex.metadataEditor`) on the project
- [Storage Object Viewer](https://docs.cloud.google.com/iam/docs/roles-permissions/storage.md.txt#storage.objectViewer) (`roles/storage.objectViewer`) on the bucket containing your import files


For more information about granting roles, see [Manage access to projects, folders, and organizations](https://docs.cloud.google.com/iam/docs/granting-changing-revoking-access.md.txt).


You might also be able to get
the required permissions through [custom
roles](https://docs.cloud.google.com/iam/docs/creating-custom-roles) or other [predefined
roles](https://docs.cloud.google.com/iam/docs/roles-overview#predefined).

### Enable APIs

To import glossaries and entry links, enable the Dataplex API in your
project.

<br />

**Roles required to enable APIs**


To enable APIs, you need the `serviceusage.services.enable` permission. If you
created the project, then you likely already have this permission through the
Owner role (`roles/owner`). Otherwise, you can get this permission through the
Service Usage Admin role (`roles/serviceusage.serviceUsageAdmin`).
[Learn how to grant roles](https://docs.cloud.google.com/iam/docs/granting-changing-revoking-access.md.txt).

[Enable the APIs](https://console.cloud.google.com/apis/enableflow?apiid=dataplex.googleapis.com,cloudresourcemanager.googleapis.com,storage.googleapis.com)

## Glossary terms versus physical entries

The structure of your catalog is based on the following components:

- **Glossary terms**: business concepts, such as annual revenue, that are organized into glossaries and categories.
- **Entries**: physical data assets, such as a BigQuery table or column.

*Glossary* import jobs create new glossary categories and terms and overwrite
any existing ones. *Entry link* import jobs create links either between glossary
terms (for example, synonyms or related terms) or between glossary terms and
physical entries (for example, between a term and a physical data asset). See
the [Manage a business glossary](https://docs.cloud.google.com/knowledge-catalog/docs/manage-glossaries.md.txt) page
for more information about structuring your catalog.

### Bulk import behavior

Business glossary and entry link imports are separate processes and have
different default quotas. See the
[Quotas and limits](https://docs.cloud.google.com/knowledge-catalog/docs/quotas-limits#quotas_for_requests)
page for more information.

Glossary bulk import jobs use `"entry_sync_mode": "FULL"` and operate in full
overwrite mode. When you run an import job for an existing glossary,
Knowledge Catalog deletes any existing categories and terms and recreates
them from your uploaded JSON files. Recreating glossary terms assigns them new
unique IDs and deletes any existing entry links attached to those terms, such as
links to BigQuery columns. To restore these relationships, you
must re-import your entry links after the glossary import job finishes. Entry
link imports only create entry new entry links, without overwriting or deleting
existing ones.

## Import glossaries using JSON files

To import glossaries, categories, and terms by uploading JSON files to
Cloud Storage, complete the following tasks.

### Create a target glossary

Create a target glossary to import metadata.

```bash
alias gcurl='curl -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/json"'

gcurl -X POST https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/glossaries?glossary_id=GLOSSARY_ID -d "$(cat<<EOF

{
   "displayName": "DISPLAY_NAME",
   "description": "DESCRIPTION"
}
EOF
)"
```

Replace the following:

- `PROJECT_ID`: the project ID in which you're creating the glossary
- `LOCATION_ID`: the location in which you want to create the glossary
- `GLOSSARY_ID`: the glossary ID
- `DISPLAY_NAME`: the display name of the glossary
- `DESCRIPTION`: the description of the glossary

### Prepare JSON files

Create a newline-delimited JSON file with glossary, categories, and terms that
you want to upload to the Cloud Storage bucket.

Use the following JSON schema to structure your import files:

```bash
   {"entry":{"name":"projects/PROJECT_NUMBER/locations/LOCATION_ID/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID/categories/CATEGORY_ID","entryType":"projects/dataplex-types/locations/global/entryTypes/glossary-category","aspects":{"dataplex-types.global.glossary-category-aspect":{"data":{}},"dataplex-types.global.overview":{"data":{"content":"CONTENT"}},"dataplex-types.global.contacts":{"data":{"identities":[{role: "steward", name: "CONTACT_DISPLAY_NAME", id: "CONTACT_EMAIL"}]}}},"parentEntry":"projects/PROJECT_NUMBER/locations/LOCATION_ID/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID","entrySource":{"resource":"projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID/categories/CATEGORY_ID","displayName":"CATEGORY_NAME","description":"CATEGORY_DESCRIPTION","ancestors":[{"name":"projects/PROJECT_NUMBER/locations/LOCATION_ID/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID","type":"projects/dataplex-types/locations/global/entryTypes/glossary"}]}}}
   {"entry":{"name":"projects/PROJECT_NUMBER/locations/LOCATION_ID/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID/terms/TERM1_ID","entryType":"projects/dataplex-types/locations/global/entryTypes/glossary-term","aspects":{"dataplex-types.global.glossary-term-aspect":{"data":{}},"dataplex-types.global.overview":{"data":{"content":"TERM1_CONTENT"}},"dataplex-types.global.contacts":{"data":{"identities":[{role: "steward", name: "CONTACT_DISPLAY_NAME", id: "CONTACT_EMAIL"}]}}},"parentEntry":"projects/PROJECT_NUMBER/locations/LOCATION_ID/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID","entrySource":{"resource":"projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID/terms/TERM1_ID","displayName":"TERM1_DISPLAY_NAME","description":"TERM1_DESCRIPTION","ancestors":[{"name":"projects/PROJECT_NUMBER/locations/LOCATION_ID/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID","type":"projects/dataplex-types/locations/global/entryTypes/glossary"},{"name":"projects/PROJECT_NUMBER/locations/LOCATION_ID/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID/categories/CATEGORY_ID","type":"projects/dataplex-types/locations/global/entryTypes/glossary-category"}]}}}
   {"entry":{"name":"projects/PROJECT_NUMBER/locations/LOCATION_ID/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID/terms/TERM2_ID","entryType":"projects/dataplex-types/locations/global/entryTypes/glossary-term","aspects":{"dataplex-types.global.glossary-term-aspect":{"data":{}},"dataplex-types.global.overview":{"data":{"content":"TERM1_CONTENT"}},"dataplex-types.global.contacts":{"data":{"identities":[{role: "steward", name: "CONTACT_DISPLAY_NAME", id: "CONTACT_EMAIL"}]}}},"parentEntry":"projects/PROJECT_NUMBER/locations/LOCATION_ID/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID","entrySource":{"resource":"projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID/terms/TERM2_ID","displayName":"TERM2_DISPLAY_NAME","description":"TERM2_DESCRIPTION","ancestors":[{"name":"projects/PROJECT_NUMBER/locations/LOCATION_ID/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID","type":"projects/dataplex-types/locations/global/entryTypes/glossary"},{"name":"projects/PROJECT_NUMBER/locations/LOCATION_ID/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/LOCATION_ID/glossaries/GLOSSARY_ID/categories/CATEGORY_ID","type":"projects/dataplex-types/locations/global/entryTypes/glossary-category"}]}}}
```

### Import glossary, categories, and terms

Import the glossary, categories, and terms into the Cloud Storage
bucket:

```bash
# Set GCURL alias
alias gcurl='curl -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/json"'

gcurl https://DATAPLEX_API/metadataJobs?metadata_job_id=JOB_ID -d "$(cat<<EOF
{
   "type":"IMPORT",
   "import_spec":{
      "log_level":"DEBUG",
      "source_storage_uri":"gs://STORAGE_BUCKET/",
      "entry_sync_mode":"FULL",
      "aspect_sync_mode":"INCREMENTAL",
      "scope":{
         "glossaries": "GLOSSARY_NAME"
      }
   }
}
EOF
)"
```

Replace `DATAPLEX_API` with the Dataplex API
endpoint of the format
`dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID`.

## Import entry links using JSON files

To bulk import entry links by uploading JSON files to Cloud Storage,
complete the following tasks.

### Prepare JSON files

Create a newline-delimited JSON file with the entry links that you want to
upload to the Cloud Storage bucket.

Use the following JSON schema to structure your import files:
Sample format for links between terms

```bash
   {"entryLink":{"name":"projects/PROJECT_NUMBER/locations/global/entryGroups/@dataplex/entryLinks/el-import-0606e3f2-8206-4f3a-aba9-32c6196f6048","entryLinkType":"projects/dataplex-types/locations/global/entryLinkTypes/synonym","entryReferences":[{"name":"projects/PROJECT_NUMBER/locations/global/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/global/glossaries/import-glossary/terms/term-1"},{"name":"projects/PROJECT_NUMBER/locations/global/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/global/glossaries/import-glossary/terms/term-2"}]}}
   {"entryLink":{"name":"projects/PROJECT_NUMBER/locations/global/entryGroups/@dataplex/entryLinks/el-import-2f7408e3-af3d-405d-81bb-861cf9ec5146","entryLinkType":"projects/dataplex-types/locations/global/entryLinkTypes/related","entryReferences":[{"name":"projects/PROJECT_NUMBER/locations/global/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/global/glossaries/import-glossary/terms/term-1"},{"name":"projects/PROJECT_NUMBER/locations/global/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/global/glossaries/import-glossary/terms/term-2"}]}}
   
```
Sample format for links between terms and data assets

```bash
   {"entryLink":{"name":"projects/PROJECT_NUMBER/locations/global/entryGroups/@dataplex/entryLinks/el-import-0606e3f2-8206-4f3a-aba9-32c6196f6048","entryLinkType":"projects/dataplex-types/locations/global/entryLinkTypes/definition","entryReferences":[{"name":"projects/PROJECT_NUMBER/locations/global/entryGroups/@dataplex/entries/projects/PROJECT_NUMBER/locations/global/glossaries/import-glossary/terms/term-1"},{"name":"projects/PROJECT_NUMBER/locations/us-central1/entryGroups/entry-group-1/entries/entry-1"}]}}
   
```

### Import links between terms as synonym or related terms

To initiate a metadata import job that establishes `synonym` or `related`
connections between your catalog terms, use the following command:

```bash
gcurl https://DATAPLEX_API/metadataJobs?metadata_job_id=JOB_ID -d "$(cat<<EOF
   {
      "type":"IMPORT",
      "import_spec":{
         "log_level":"DEBUG",
         "source_storage_uri":"gs://STORAGE_BUCKET/",
         "entry_sync_mode":"FULL",
         "aspect_sync_mode":"INCREMENTAL",
         "scope":{
            "entry_groups":[  "projects/PROJECT_ID/locations/LOCATION_ID/entryGroups/@dataplex"
            ],
            "entry_link_types":[
               "projects/dataplex-types/locations/global/entryLinkTypes/synonym",
               "projects/dataplex-types/locations/global/entryLinkTypes/related"
            ],
            "referenced_entry_scopes":[
               "PROJECT_IDS"
            ]
         }
      }
   }
EOF
)"
```

Replace `DATAPLEX_API` with
`dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID`.

### Import links between terms and specific data assets or columns

To create a link between glossary terms and data assets, run the import
job for each entry group to which the entry for the data asset belongs. All
definition entry links are created in this entry group.

> [!NOTE]
> **Note:** Make sure the import file in the Cloud Storage bucket contains a file for links in one entry group. To create a link between a term and entry for the data asset in another entry group, provide a new file in the Cloud Storage bucket.

```bash
   gcurl https://DATAPLEX_API/metadataJobs?metadata_job_id=JOB_ID -d "$(cat<<EOF
   {
   "type":"IMPORT",
   "import_spec":{
      "log_level":"DEBUG",
      "source_storage_uri":"gs://STORAGE_BUCKET/",
      "entry_sync_mode":"FULL",
      "aspect_sync_mode":"INCREMENTAL",
      "scope":{
         "entry_groups":[  "projects/PROJECT_ID/locations/ENTRY_GROUP_LOCATION_ID/entryGroups/@dataplex"
         ],
         "entry_link_types":[
            "projects/dataplex-types/locations/global/entryLinkTypes/definition"
         ],
         "referenced_entry_scopes":[
            "PROJECT_IDS"
         ]
      }
   }
   }
EOF
)"
```

Replace `DATAPLEX_API` with
`dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID`.

## Monitor the import job status

1. To get the status of the `import` operation, run the following command:

   ```bash
   gcurl https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/operations/operation-OPERATION_ID
   ```

   Replace `OPERATION_ID` with the ID of the operation.
2. To get the status of the metadata job, run the following command:

   ```bash
   gcurl -X GET https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/metadataJobs/JOB_ID
   ```

## What's next

- Learn how to [manage business glossaries in Knowledge Catalog](https://docs.cloud.google.com/knowledge-catalog/docs/manage-glossaries.md.txt).
- Learn how to [import glossaries from a Google Sheet](https://docs.cloud.google.com/knowledge-catalog/docs/import-glossaries-sheets.md.txt).
- Learn more about [metadata management](https://docs.cloud.google.com/knowledge-catalog/docs/metadata-overview.md.txt).