Credential vending overview

Credential vending for the Lakehouse runtime catalog lets you delegate storage access and apply fine-grained permissions to your data files. As part of Lakehouse for Apache Iceberg, this capability lets you manage Identity and Access Management (IAM) policies at the table level for tables stored in Cloud Storage.

You can use the gcloud CLI to get and set these policies to control access to your resources.

How credential vending works

When you use credential vending, the query processing sequence changes slightly to enforce policies before data is read:

  1. Request: A user submits a SQL query to a supported engine (for example, Apache Spark or BigQuery).
  2. Metadata lookup: The engine sends a request to the Lakehouse runtime catalog to resolve the table.
  3. Authentication and policy: The catalog authenticates the user and checks their IAM permissions on Google Cloud's Lakehouse resources.
  4. Response: Because credential vending is enabled, the catalog returns the metadata and a short-lived storage token (downscoped storage credentials) to the engine.
  5. Read: The engine uses this token to read the specific authorized files directly from Cloud Storage.
  6. Compute: The engine processes the data and returns the results.

Supported engines

In order to use credential vending with query engines, your Lakehouse Iceberg REST catalog must be configured to support credential vending.

  • Open source engines: Supported engines like Apache Spark and Trino use short-lived storage tokens vended by the catalog. Your client application must specify support for credential vending in the X-Iceberg-Access-Delegation header.
  • BigQuery: BigQuery uses vended credentials for Cloud Storage access instead of end-user credentials.

Required service account permissions

When credential vending is enabled, ensure the following service accounts have the necessary roles:

  • Auto-provisioned Lakehouse runtime catalog service account: Must be explicitly granted the Storage Object User role (roles/storage.objectUser) on the target Cloud Storage bucket. By default, this service account is created with viewer-only access. Without object admin access, vended credentials cannot perform storage writes. (Note: If configuring via the console UI, clicking Set bucket permissions verifies this role. For gcloud, Terraform, or API setups, this role must be granted manually).
  • Query engine service accounts: Service accounts running query engine jobs (such as Managed Service for Apache Spark, Managed Service for Apache Spark, or Dataflow) require the BigLake Editor role (roles/biglake.editor) at the project level in order to obtain vended credentials with write scope.

What's next