Overview of Apache Iceberg tables

Apache Iceberg tables, managed by the Lakehouse runtime catalog and the Iceberg REST catalog endpoint, are open and interoperable Iceberg tables adhering to the open source Apache Iceberg Table specification, which can be read and written by using any Iceberg-compatible processing engine or BigQuery.

In this document, these tables are referred to as "Lakehouse Iceberg tables" or "Iceberg tables".

You can opt in and enable the following capabilities by configuring table options:

  1. Read/write interoperability (Preview): Seamless read and write support for multiple engines, including BigQuery, Managed service for Apache Spark, Iceberg-compatible engines such as open source engines (Apache Spark, Apache Flink, and Trino), and third-party engines (such as Snowflake).
  2. Automatic table management (Preview): Automatic table management (storage optimization) such as compaction and garbage collection.

In addition to the preceding capabilities, the following capabilities are also supported by default for Lakehouse Iceberg tables (Preview).

  1. BigQuery Data definition language (Preview): You can also create or update Iceberg tables by using BigQuery DDL statements (for example, CREATE TABLE, ALTER TABLE, and DROP TABLE), in addition to creating or updating tables by using Iceberg-compatible engines such as Spark, Flink, and Trino. After the tables are created, they are available for reads and writes in those engines and in BigQuery, and are part of the same catalog and namespace construct, managed by the Lakehouse runtime catalog.
  2. Credential vending support for BigQuery (Preview): BigQuery supports using credential vending for authentication when reading or writing Iceberg tables in the Lakehouse Runtime Catalog. You can set credential vending at the catalog level.

Management actions

You can perform the following management actions on your Apache Iceberg tables:

  • Create a table: Create an Apache Iceberg table within a catalog namespace using the Google Cloud console, Spark, Trino, gcloud, BigQuery (Preview), or the Iceberg REST Catalog API (CreateIcebergTable).
  • List tables: View table identifiers within a namespace using the Google Cloud console, Spark, Trino, gcloud, BigQuery (Preview), or the Iceberg REST Catalog API (ListIcebergTableIdentifiers).
  • Get table details: Inspect table schema, properties, and credentials using the Google Cloud console, Spark, Trino, gcloud, BigQuery (Preview), or the Iceberg REST Catalog API (GetIcebergTable, LoadIcebergTableCredentials).
  • Insert data: Append data rows to your Iceberg tables using Spark, Trino, or BigQuery (Preview).
  • Query a table: Execute queries against your Iceberg tables from Spark, Trino, or BigQuery (Preview) using four-part table naming.
  • Modify data with DML: Update, delete, or merge data rows in your Iceberg tables using DML statements from BigQuery (Preview), Spark, or Trino.
  • Alter a table: Evolve table schema and update metadata properties using the Google Cloud console, Spark, Trino, gcloud, BigQuery (Preview), or the Iceberg REST Catalog API (UpdateIcebergTable).
  • Configure table options: Configure properties to enable BigQuery (Preview) DML and automatic table management.
  • Manage table ACLs: View and update IAM policies on your Iceberg tables to control access for specific principals (get-iam-policy, set-iam-policy).
  • Upgrade Iceberg V1 tables to V2: Upgrade your existing Iceberg V1 tables to the supported V2 format.
  • Use Binary Deletion Vectors in Iceberg V3 tables: Enable binary deletion vectors for optimized deletion performance in Iceberg V3 tables.
  • Delete a table: Drop a table registration from the catalog without purging underlying storage files using the Google Cloud console, Spark, Trino, gcloud, BigQuery (Preview), or the Iceberg REST Catalog API (DeleteIcebergTable).

What's next