Create a catalog

Creating a catalog (such as an Apache Iceberg REST catalog or Apache Hive catalog) establishes a management endpoint within the Lakehouse runtime catalog. This endpoint points to an underlying Cloud Storage bucket, providing a metadata layer that lets query engines and open-source workloads interact directly with your tables.

When creating your catalog for Lakehouse for Apache Iceberg, you can choose between end-user credentials or credential vending mode for storage access delegation.

Before you begin

  1. Verify that billing is enabled for your Google Cloud project.

  2. Enable the BigLake API.

    Roles required to enable APIs

    To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains the serviceusage.services.enable permission. Learn how to grant roles.

    Enable the API

Required roles

To get the permissions that you need to create a catalog, ask your administrator to grant you the following IAM roles:

  • All:
  • Auto-provisioned Lakehouse runtime catalog service account in credential vending mode: Storage Object User (roles/storage.objectUser) on the target Cloud Storage bucket. After creating the catalog, explicitly grant the Storage Object User role (roles/storage.objectUser) on your storage bucket to your catalog's auto-provisioned Lakehouse runtime catalog service account.

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Create a catalog

Create a catalog endpoint.

Console

  1. In the Google Cloud console, open the Lakehouse page.

    Go to Lakehouse

  2. Select Create catalog. The Create catalog page opens.

  3. Select the catalog configuration:

    • To create an Apache Iceberg REST catalog, select Iceberg REST catalog.
    • To create an Apache Hive catalog, select Hive catalog.

  4. Configure the catalog:

    • If you choose Iceberg REST catalog, select Cloud Storage bucket for the catalog type, and select the Cloud Storage bucket to use with your catalog. You can only have one catalog per bucket, and the catalog name matches the bucket name.
    • If you choose Hive catalog, enter a unique catalog name and select the associated Cloud Storage path.

  5. For Authentication method, select either End-user credentials or Credential vending mode.

    If you select Credential vending mode, the auto-provisioned Lakehouse runtime catalog service account requires the explicit Storage Object User role (roles/storage.objectUser) on all associated Cloud Storage buckets. By default, it has no access entirely. Without this role, vended credentials do not have sufficient scope to perform storage writes.

  6. Select Create.

    Your catalog is created and the Catalog details page opens.

  7. Under Authentication method, select Set bucket permissions.

  8. In the dialog, select Confirm.

    This verifies that your catalog's service account has the Storage Object User role (roles/storage.objectUser) on all associated storage buckets. If you create catalogs using gcloud or Terraform, you must grant this role manually.

gcloud

To create a catalog using gcloud, run the gcloud biglake iceberg catalogs create command.

Create a BigLake catalog

To create a BigLake catalog, set --catalog-type to biglake and specify the default warehouse location:

gcloud biglake iceberg catalogs create CATALOG_ID \
    --project="PROJECT_ID" \
    --catalog-type="biglake" \
    --default-location="DEFAULT_LOCATION" \
    [--restricted-locations="RESTRICTED_LOCATIONS"] \
    [--credential-mode="CREDENTIAL_MODE"] \
    [--primary-location="PRIMARY_LOCATION"]

Create a Cloud Storage bucket catalog

To create a Cloud Storage bucket catalog, set --catalog-type to gcs-bucket:

gcloud biglake iceberg catalogs create CATALOG_ID \
    --project="PROJECT_ID" \
    --catalog-type="gcs-bucket" \
   [--credential-mode="CREDENTIAL_MODE"]

Replace the following:

  • CATALOG_ID: the ID of your Lakehouse runtime catalog. For Cloud Storage bucket catalogs, this must match the name of the Cloud Storage bucket.
  • PROJECT_ID: your Google Cloud project ID.
  • DEFAULT_LOCATION: the base Cloud Storage path for the catalog data and metadata, in the format gs://my-bucket/path.
  • RESTRICTED_LOCATIONS: (Optional) a comma-separated list of additional allowed storage buckets or paths.
  • CREDENTIAL_MODE: the authentication method. Use end-user for End-user credentials or vended-credentials for Credential vending mode.
  • PRIMARY_LOCATION: (Optional) the primary region for the catalog (such as US or EU) to ensure compatibility with BigQuery.

REST

To create a catalog management endpoint using the REST API, make a POST request to the CreateIcebergCatalog endpoint:

POST /iceberg/v1/restcatalog/extensions/projects/PROJECT_ID/catalogs?icebergCatalogId=CATALOG_ID

The request body must contain an IcebergCatalog JSON payload defining the catalog configuration, such as the underlying Cloud Storage bucket warehouse and authentication mode.

Replace the following:

  • PROJECT_ID: your Google Cloud project ID.
  • CATALOG_ID: the ID of your Lakehouse runtime catalog.