In borderless Lakehouse, a cross-cloud connection lets you access
remote data from external platforms without moving files between cloud
environments. You can then query your federated tables from BigQuery,
Managed Service for Apache Spark, or open-source Apache Spark.

In addition to analytical queries, you can use your federated data for
AI-driven insights and governance:

- [**Conversational analytics**](https://docs.cloud.google.com/lakehouse/docs/conversational-analytics.md.txt): build specialized agents grounded in your data sources, including federated tables, to analyze data across clouds from a single conversation.
- [**Knowledge Catalog**](https://docs.cloud.google.com/knowledge-catalog/docs/metadata-overview.md.txt#supported-sources): profile data and generate insights across your federated data sources.

For deeper analysis, you can author conversational analytics agents that use projects,
datasets, tables, views, graphs, and user-defined functions. These agents work
across BigQuery standard tables, Apache Iceberg tables managed
by Lakehouse, and Lakehouse sources such as Databricks
Unity Catalog, AWS Glue, SAP BDC, and Salesforce.

## Before you begin

Before you can query your data, you must complete the following:

1. Ensure that you have data within your remote catalog.
2. Set up a cross-cloud connection for [AWS Glue](https://docs.cloud.google.com/lakehouse/docs/set-up-cross-cloud-connection-aws-glue.md.txt), [Databricks Unity Catalog](https://docs.cloud.google.com/lakehouse/docs/set-up-cross-cloud-connection-databricks.md.txt), [Snowflake Horizon Catalog](https://docs.cloud.google.com/lakehouse/docs/set-up-cross-cloud-connection-snowflake.md.txt), [Workday Data Lake](https://docs.cloud.google.com/lakehouse/docs/set-up-cross-cloud-connection-workday.md.txt), or [SAP Business Data Cloud](https://docs.cloud.google.com/lakehouse/docs/set-up-cross-cloud-connection-sap-bdc.md.txt).

### Required roles


To get the permissions that
you need to query federated data,

ask your administrator to grant you the
following IAM roles on your project:

- Query data in BigQuery: [BigQuery Data Viewer](https://docs.cloud.google.com/iam/docs/roles-permissions/bigquery.md.txt#bigquery.dataViewer) (`roles/bigquery.dataViewer`)
- Run BigQuery jobs: [BigQuery Job User](https://docs.cloud.google.com/iam/docs/roles-permissions/bigquery.md.txt#bigquery.jobUser) (`roles/bigquery.jobUser`)
- Discover and read table metadata in Lakehouse catalogs: [BigLake Viewer](https://docs.cloud.google.com/iam/docs/roles-permissions/biglake.md.txt#biglake.viewer) (`roles/biglake.viewer`)


For more information about granting roles, see [Manage access to projects, folders, and organizations](https://docs.cloud.google.com/iam/docs/granting-changing-revoking-access.md.txt).


You might also be able to get
the required permissions through [custom
roles](https://docs.cloud.google.com/iam/docs/creating-custom-roles) or other [predefined
roles](https://docs.cloud.google.com/iam/docs/roles-overview#predefined).

## Query data

After you set up federation, you can query your remote data using standard SQL
in BigQuery or Apache Spark in [Managed Service for Apache Spark](https://docs.cloud.google.com/dataproc/docs).

*Lakehouse* handles the metadata translation and secure data
access, which lets you treat remote Apache Iceberg tables as if they were local
to your Google Cloud environment.

### Query from BigQuery

To query federated Apache Iceberg tables, use standard BigQuery
SQL. The table path follows a 4-part structure:
`project.federated_catalog.namespace.table`. Credential vending, CCI transit
routing, and cross-cloud caching (when enabled on the catalog) are automatically
handled by Google Cloud.

```sql
SELECT
  user_id,
  action,
  COUNT(*) as total_actions
FROM `PROJECT_ID.FEDERATED_CATALOG_NAME.NAMESPACE_NAME.TABLE_NAME`
WHERE event_date >= '2026-04-01'
GROUP BY 1, 2;
```

Replace the following:

- `PROJECT_ID`: your Google Cloud project ID.
- `FEDERATED_CATALOG_NAME`: the name of the federated catalog.
- `NAMESPACE_NAME`: the namespace within the catalog.
- `TABLE_NAME`: the name of the table.
- `REGION`: the Google Cloud region. For example, `us-east4`.

You can also run the query using the `bq` command-line tool:

```bash
bq --location="REGION" --project_id="PROJECT_ID" query --use_legacy_sql=false \
  "SELECT * FROM \`PROJECT_ID.FEDERATED_CATALOG_NAME.NAMESPACE_NAME.TABLE_NAME\` LIMIT 10"
```

### Query from Managed Service for Apache Spark

Submit a [PySpark batch workload](https://docs.cloud.google.com/dataproc-serverless/docs/quickstarts/spark-batch#submit_a_spark_batch_workload) to Managed Service for Apache Spark with
[credential vending](https://docs.cloud.google.com/lakehouse/docs/credential-vending.md.txt) enabled using
`X-Iceberg-Access-Delegation=vended-credentials`. Spark will use the short-lived
scoped vended credentials to connect to S3 securely, all without needing to
manage separate AWS credentials or S3 connectors.

1. Enable outbound connectivity for Managed Service for Apache Spark.

   Managed Service for Apache Spark cannot connect to AWS S3 with its default
   [network configuration](https://docs.cloud.google.com/dataproc/docs/concepts/network-serverless). You must provision a Cloud Router and
   Cloud NAT.

   ```bash
   gcloud compute routers create lakehouse-router \
     --network=NETWORK_NAME \
     --region=REGION

   gcloud compute routers nats create lakehouse-nat \
     --router=lakehouse-router \
     --auto-allocate-nat-external-ips \
     --nat-all-subnet-ip-ranges \
     --region=REGION
   ```

   Replace the following:
   - `NETWORK_NAME`: the network for the Managed Service for Apache Spark batch workload (for example, `default`).
   - `REGION`: the region for the Managed Service for Apache Spark batch workload.
2. Create a PySpark application file and run the PySpark job.

   ```python
   from pyspark.sql import SparkSession
   spark = SparkSession.builder.appName("CATALOG_NAME").getOrCreate()

   df = spark.table("CATALOG_NAME.NAMESPACE_NAME.TABLE_NAME")
   df.show(10, truncate=False)
   ```

   Upload this to Cloud Storage at `PYSPARK_FILE`.

   > [!NOTE]
   > **Note:** The properties and parameters required for the Spark job might vary depending on your data source (For example, Databricks or AWS Glue) and underlying storage. For a list of parameters, see [Spark Configuration Parameters](https://docs.cloud.google.com/lakehouse/docs/query-remote-data.md.txt#spark-parameters).

   ```bash
   gcloud dataproc batches submit pyspark PYSPARK_FILE \
       --project=PROJECT_ID \
       --region=REGION \
       --version=RUNTIME_VERSION \
       --properties="\
       spark.sql.defaultCatalog=CATALOG_NAME,\
       spark.sql.catalog.CATALOG_NAME=org.apache.iceberg.spark.SparkCatalog,\
       spark.sql.catalog.CATALOG_NAME.type=rest,\
       spark.sql.catalog.CATALOG_NAME.uri=https://biglake.googleapis.com/iceberg/v1/restcatalog,\
       spark.sql.catalog.CATALOG_NAME.warehouse=bl://projects/PROJECT_ID/catalogs/FEDERATED_CATALOG_NAME,\
       spark.sql.catalog.CATALOG_NAME.header.x-goog-user-project=PROJECT_ID,\
       spark.sql.catalog.CATALOG_NAME.rest.auth.type=org.apache.iceberg.gcp.auth.GoogleAuthManager,\
       spark.sql.catalog.CATALOG_NAME.io-impl=IO_IMPL,\
       spark.sql.catalog.CATALOG_NAME.header.X-Iceberg-Access-Delegation=vended-credentials,\
       spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions"
   ```

   Replace the following:
   - `NAMESPACE_NAME`: the namespace in the federated catalog.
   - `TABLE_NAME`: the name of the table in the federated catalog.
   - `CATALOG_NAME`: a name for the local Spark catalog (for example, `my_catalog`).
   - `PYSPARK_FILE`: the `gs://` Cloud Storage path to your PySpark application file.
   - `REGION`: the region for the Managed Service for Apache Spark batch workload.
   - `RUNTIME_VERSION`: the Managed Service for Apache Spark runtime version, for example `2.3`.
   - `PROJECT_ID`: the project that is billed for using the Apache Iceberg REST catalog endpoint.
   - `FEDERATED_CATALOG_NAME`: the name of the federated catalog.
   - `IO_IMPL`: the FileIO implementation matching your underlying storage.

   ### Spark Configuration Parameters

   The following table lists common parameters required for all connections:

   | Parameter | Description |
   |:---|:---|
   | `spark.sql.defaultCatalog` | The default catalog name (for example, `CATALOG_NAME`). |
   | `spark.sql.catalog.CATALOG_NAME` | The catalog implementation class. Set to `org.apache.iceberg.spark.SparkCatalog`. |
   | `spark.sql.catalog.CATALOG_NAME.type` | The catalog backend type. Set to `rest` for the Iceberg REST catalog. |
   | `spark.sql.catalog.CATALOG_NAME.uri` | The URI of the REST catalog endpoint. Set to `https://biglake.googleapis.com/iceberg/v1/restcatalog`. |
   | `spark.sql.catalog.CATALOG_NAME.warehouse` | The warehouse location path for the federated catalog. Set to `bl://projects/PROJECT_ID/catalogs/FEDERATED_CATALOG_NAME`. |
   | `spark.sql.catalog.CATALOG_NAME.header.x-goog-user-project` | The Google Cloud project ID used for billing and quota attribution. Set to `PROJECT_ID`. |
   | `spark.sql.extensions` | The Spark session extensions for Iceberg SQL syntax and features. Set to `org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions`. |

   The following table lists authentication parameters:

   > [!NOTE]
   > **Note:** Personal Access Token (PAT) authentication (`spark.sql.catalog.CATALOG_NAME.token` and `spark.sql.catalog.CATALOG_NAME.oauth2-server-uri`) and custom authentication type (`spark.sql.catalog.CATALOG_NAME.rest.auth.type`) are mutually exclusive. When both are set, PAT takes priority and the authentication type is assumed to be `BEARER`.

   | Parameter | Description |
   |:---|:---|
   | `spark.sql.catalog.CATALOG_NAME.rest.auth.type` | The custom authentication manager class. Set to `org.apache.iceberg.gcp.auth.GoogleAuthManager` for OAuth flow authentication. |
   | `spark.sql.catalog.CATALOG_NAME.oauth2-server-uri` | The OAuth2 token server endpoint URI. Set to `https://oauth2.googleapis.com/token` for Personal Access Token (PAT) authentication. |
   | `spark.sql.catalog.CATALOG_NAME.token` | The Bearer token or Personal Access Token (PAT). Typically set to `$(gcloud auth application-default print-access-token)` for PAT authentication. |

   The following table lists unique parameters based on your storage provider (`IO_IMPL`):

   | Storage | `spark.sql.catalog.CATALOG_NAME.io-impl` | Notes |
   |:---|:---|:---|
   | **Amazon S3** | `org.apache.iceberg.aws.s3.S3FileIO` | Requires credential vending (`X-Iceberg-Access-Delegation=vended-credentials`) if enabled. Additional parameter: `spark.sql.catalog.CATALOG_NAME.s3.region` (for a list of regions, see [Amazon S3 endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/s3.html#s3_region)). |
   | **Cloud Storage** | `org.apache.iceberg.gcp.gcs.GCSFileIO` | Additional parameter: `spark.sql.catalog.CATALOG_NAME.gcs.oauth2.refresh-credentials-endpoint`=`https://oauth2.googleapis.com/token`. |
   | **Azure Blob Storage** | `org.apache.iceberg.azure.adlsv2.ADLSFileIO` | No additional storage parameters required. |

   For Snowflake, you may encounter issues when querying `STRING` columns because
   they are automatically storage-optimized. You can resolve this in one of two
   ways:
   - **Option 1: Disable vectorization in Spark** : Add the following Spark configuration properties to the `--properties` flag:
   - `spark.sql.iceberg.vectorization.enabled=false`
   - `spark.sql.catalog.CATALOG_NAME.table-override.read.parquet.vectorization.enabled=false`

   - **Option 2: Change serialization policy in Snowflake** : Change the storage serialization policy to `COMPATIBLE` for the table in Snowflake.

### Monitor cache usage and egress cost savings

When you enable cross-cloud caching on a federated catalog or
BigQuery connection, Lakehouse caches retrieved
file blocks locally within the Google Cloud query execution region.
Subsequent queries, including different queries targeting the same file
blocks, read directly from the local cache instead of refetching the files from
the remote cloud.

To verify cache hit rates and measure your egress cost savings, inspect the job
details in the BigQuery console or retrieve the
[`JobStatistics2` query job statistics](https://docs.cloud.google.com/bigquery/docs/reference/rest/v2/Job.md.txt#JobStatistics2)
using the BigQuery API or bq command-line tool:

```bash
bq show --format=prettyjson --job=true JOB_ID
```

Replace `JOB_ID` with the ID of your
BigQuery query job.

In the JSON output under `statistics.query.objectStorageStats` (or
`object_storage_stats` in the proto API), the job statistics return a list with
an entry for each cloud provider accessed during execution. Each entry contains
the following fields:

- **`cloudProvider`** (`cloud_provider`): the cloud provider that hosts the object storage (for example, `AWS` or `AZURE`).
- **`cacheBytesRead`** (`cache_bytes_read`): the total bytes read from the local Google Cloud cache, avoiding a remote object storage read.
- **`objectStorageBytesRead`** (`object_storage_bytes_read`): the total bytes read directly from the cloud provider's object storage.

For more information about caching concepts and data residency considerations,
see [Intelligent caching](https://docs.cloud.google.com/lakehouse/docs/about-cross-cloud-data-access.md.txt#intelligent-caching).

## What's next

- [Use VPC Service Controls with federated catalog endpoints](https://docs.cloud.google.com/lakehouse/docs/vpc-sc-federated-catalogs.md.txt).
- [Supported regions and capabilities](https://docs.cloud.google.com/lakehouse/docs/regions-capabilities-cross-cloud-connections.md.txt).