Compare table types

Choosing the right table architecture is critical for maximizing performance, reducing costs, and ensuring data access across your analytics tools. This page explains the different table types and serving endpoints available in Lakehouse for Apache Iceberg, helping you choose the best option based on your write engines, read requirements, and management control needs.

Table formats by catalog or engine

Select a catalog or engine to learn about its supported table formats, metastore configuration, storage optimization capabilities, and engine interoperability.

Lakehouse runtime catalog

The Lakehouse runtime catalog manages Apache Iceberg tables through the Iceberg REST catalog endpoint and provides seamless read-write interoperability across Iceberg compatible engines (Spark, Flink, Trino) and BigQuery, while being backed by the industry-standard Iceberg REST catalog interface.

Supported table formats

Apache Iceberg V2 tables (GA) and V3 tables (Preview) are supported. Iceberg V1 tables aren't supported. Before you use existing V1 tables with Lakehouse for Apache Iceberg, you must upgrade them to a supported version. For more information, see Upgrade Iceberg V1 tables to V2.

Key features include:

  • Metastore: Lakehouse runtime catalog.
  • Storage: Cloud Storage.
  • Storage optimization: Managed by you, or optionally by Google (Preview).
  • Read and write access:
    • Open source engines: read and write (GA)
    • BigQuery: read/write (Preview)
  • Use cases: Open lakehouse with high-performance, enterprise-grade storage for advanced analytics, streaming, and AI.

Hive metastore

The Lakehouse runtime catalog manages Apache Hive tables through an Apache Hive metastore (HMS) endpoint optimized for Apache Spark ExternalCatalog compatibility, letting you seamlessly share data across Apache Spark, Apache Hive, and BigQuery. You create these tables from open source engines and store them in Cloud Storage. This option is best if you want your ETL workflow to be managed by open source engines without needing a separate self-hosted Hive metastore, and only require read access from BigQuery.

Tables managed by the Hive metastore endpoint are standard Apache Hive and Spark tables (using Hive SerDes or Spark data sources), not Apache Iceberg tables. To create and manage Apache Iceberg tables in the Lakehouse runtime catalog, use the Iceberg REST catalog endpoint instead.

Key features include:

  • Metastore: Lakehouse runtime catalog (through custom IMetastoreClient).
  • Storage: Cloud Storage (supporting formats like Parquet, ORC, and Avro).
  • Storage optimization: Managed by you or a third party.
  • Read and write access:
    • Open source engines (Spark and Hive): Read and write.
    • BigQuery: read-only.
  • Use cases: Migrating existing Spark and Hive workloads to a fully managed, serverless metastore on Google Cloud.

BigQuery

BigQuery supports Apache Iceberg managed tables, native tables, and external tables.

  • Apache Iceberg managed tables: These are Apache Iceberg tables that you create and manage from BigQuery and store in Cloud Storage. While they can be read by open source engines, BigQuery is the engine that manages the metadata and writes to them. This option is best if you want your workflow to be fully managed by BigQuery.

  • Native tables: These are native BigQuery tables. They are fully managed and offer the most advanced analytics and management features. This option is best for non-Iceberg workloads.

  • External tables: These tables are BigQuery-specific constructs for data stored in Cloud Storage, Amazon S3, or Azure Blob Storage. The data and metadata are self-managed, and BigQuery only has read access. Choose this option for data you want to manage in a third-party catalog or storage directly.

Table formats by product

Use the following chart to compare table types between the Lakehouse runtime catalog and BigQuery.

Lakehouse

Apache Iceberg (GA) Apache Hive (Preview)
Metastore Lakehouse runtime catalog Lakehouse runtime catalog
Storage Cloud Storage Cloud Storage
Storage optimization Customer, third-party managed, or Google managed (Preview) Customer or third-party managed
Read/write Open source engines (read/write)

BigQuery (read/write). Preview 		Open source engines (read/write)

BigQuery (read-only)
Advanced operations None None
Use cases Open lakehouse Migrate existing Spark and Hive workloads to a fully managed, serverless metastore

BigQuery

Apache Iceberg managed tables External tables Standard tables
Metastore BigQuery External or self-hosted metastore BigQuery
Storage Cloud Storage Cloud Storage / Amazon S3 / Azure BigQuery
Storage optimization Google managed Customer or third-party managed Google managed
Read/write Open source engines (read-only with Iceberg libraries, read/write interoperability with BigQuery Storage API)

BigQuery (read/write)

 Open source engines (read/write)

BigQuery (read-only) 		Open source engines (read/write interoperability with BigQuery Storage API)

BigQuery (read/write)
Advanced operations High-throughput streaming with BigQuery Storage Write API, Change Data Capture (CDC), and multi-statement transactions None High-throughput streaming with BigQuery Storage Write API, Change Data Capture (CDC), and multi-statement transactions
Use cases Open lakehouse with high-performance, enterprise-grade storage for advanced analytics, streaming, and AI Staging tables for BigQuery loads, legacy query-only tables Enterprise-grade storage for advanced analytics, streaming, and AI

What's next