Insert data into a table

Inserting data appends new records and data files to your Apache Iceberg table.

Before you begin

  1. Verify that billing is enabled for your Google Cloud project.

  2. Enable the BigLake API.

    Roles required to enable APIs

    To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains the serviceusage.services.enable permission. Learn how to grant roles.

    Enable the API

  3. Set up the Lakehouse runtime catalog with the Apache Iceberg REST catalog endpoint.

Required roles

To get the permissions that you need to insert data into a table, ask your administrator to grant you the following IAM roles on your project and storage bucket:

  • Write table data in credential vending mode: BigLake Editor (roles/biglake.editor) - the project
  • Write table data in non-credential vending mode:
    • BigLake Editor (roles/biglake.editor) - the project
    • Storage Object User (roles/storage.objectUser) - the Cloud Storage bucket

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Table capabilities and support

When using tables in the Lakehouse runtime catalog, it's helpful to understand the different table types and their opt-in capabilities. To learn more about using Apache Iceberg tables specifically, see Overview of Apache Iceberg tables.

Supported Iceberg tables

Only Apache Iceberg V2 (GA) and V3 (Preview) tables are supported. Iceberg V1 tables aren't supported. To upgrade existing V1 tables, see Upgrade Iceberg V1 tables to V2.

Use table options (Preview)

You can opt in to use BigQuery managed capabilities, such as BigQuery Data Manipulation Language (DML) and automatic table management, by configuring specific table properties. These features are enabled in different ways depending on where the table is created:

  • From BigQuery: BigQuery DML and automatic table management are enabled by default.
  • From open source engines: To opt-in, you must explicitly configure table properties. See Configure table options for more information.

Insert data into the table

Insert sample data into the table.

To insert data from BigQuery, the table must have BigQuery DML enabled (gcp.biglake.bigquery-dml.enabled = true). See Configure table options for detailed instructions.

Spark 

spark.sql("INSERT INTO TABLE_NAME VALUES (1, \"first row\"), (2, \"second row\"), (3, \"third row\");")

To enable read/write interoperability and table management (preview), add the properties to the TBLPROPERTIES clause:

TBLPROPERTIES (
  'gcp.biglake.bigquery-dml.enabled' = true,
  'gcp.biglake.table-management.enabled' = true
)

Trino 

INSERT INTO TABLE_NAME VALUES (1, 'first row'), (2, 'second row'), (3, 'third row');

To enable read/write interoperability and table management (Preview), add these properties to the WITH clause:

WITH (
  "gcp.biglake.bigquery-dml.enabled" = 'true',
  "gcp.biglake.table-management.enabled" = 'true'
)

BigQuery

To insert data into an Apache Iceberg table in the Lakehouse runtime catalog from BigQuery, the table must have BigQuery DML enabled. Use the following GoogleSQL INSERT statement:

INSERT INTO `PROJECT_ID.CATALOG_ID.NAMESPACE.TABLE_NAME` VALUES (1, "foo");

Replace the following:

  • PROJECT_ID: your Google Cloud project ID.
  • CATALOG_ID: your Lakehouse runtime catalog ID.
  • NAMESPACE: your Iceberg namespace name.
  • TABLE_NAME: the name of your Iceberg table.

What's next