Set up cross-cloud Lakehouse for Snowflake

This document describes how to set up a cross-cloud Lakehouse to query data from a Snowflake catalog (Snowflake Horizon) directly within Google Cloud. This capability unifies your data analytics by integrating your external data sources with your existing Google Cloud environment.

Afterward, you can use Lakehouse to manage access to your federated data.

Before you begin

  1. Review the Lakehouse overview to understand how Lakehouse manages access to data.
  2. Read About cross-cloud Lakehouse to understand how it works.
  3. Review the supported catalogs to verify external location requirements and supported configurations.
  4. Understand how to use regional Secret Manager secrets. This is required to set up a cross-cloud Lakehouse with Snowflake using secret-based authentication.
  5. If using secret-based authentication, generate a Personal Access Token (PAT) within your Snowflake Horizon environment that has read access to the target catalog. This process is outside the scope of this documentation.
  6. If using Workload Identity Federation, ensure you have access to the Snowflake account UI with ACCOUNTADMIN privileges to provision service users.
  7. Optional: If you plan to route queries over a private interconnect between your Google Cloud VPC and your remote cloud provider's VPC (for example, AWS), ensure that you have an active account with your remote provider, provision a Dedicated Cross-Cloud Interconnect or Partner Cross-Cloud Interconnect, establish BGP sessions with your Cloud Router, and verify that you have the required Identity and Access Management (IAM) permissions in both cloud environments.
  8. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  9. Verify that billing is enabled for your Google Cloud project.

  10. Enable the BigLake, Secret Manager APIs.

    Roles required to enable APIs

    To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

    Enable the APIs

  11. Verify that billing is enabled for your Google Cloud project.

  12. Enable the BigLake, Secret Manager APIs.

    Roles required to enable APIs

    To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

    Enable the APIs

Required roles

To get the permissions that you need to set up cross-cloud Lakehouse, ask your administrator to grant you the following IAM roles on your project:

  • Manage Lakehouse catalogs: BigLake Admin (roles/biglake.admin)
  • Manage secrets: Secret Manager Admin (roles/secretmanager.admin)
  • Route traffic over private interconnect: Compute Network Admin (roles/compute.networkAdmin), Service Directory Viewer (roles/servicedirectory.viewer), and Service Directory PSC Authorized Service (roles/servicedirectory.pscAuthorizedService)

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Supported catalog details

This guide provides instructions for setting up cross-cloud Lakehouse with a Snowflake catalog (Snowflake Horizon) on Amazon Web Services (AWS) and Google Cloud. For detailed information regarding external location requirements and supported configurations, see Supported catalogs.

Limitations and considerations

This section lists the limitations and considerations for using cross-cloud Lakehouse.

  • Supported Cloud Providers: Using a private interconnect with your cross-cloud Lakehouse is supported with the following remote cloud providers: Amazon Web Services (AWS). You can use either a Dedicated Cross-Cloud Interconnect or a Partner Cross-Cloud Interconnect.
  • Network routing: If a private interconnect (such as Dedicated CCI or Partner CCI) is not configured, queries route over the public internet. This can result in higher egress fees from your remote cloud provider and less predictable performance.
  • Data freshness: The --refresh-interval flag for the federated catalog determines how often metadata is synchronized. A shorter interval provides fresher data but can incur additional API costs from the remote catalog provider.
  • Iceberg Metrics Reporting: Iceberg Metrics Reporting is not available for federated catalogs. Set the rest-metrics-reporting-enabled property to false in your Iceberg client when accessing a federated catalog.

General workflow

To set up and use cross-cloud Lakehouse, follow these general steps:

  • Set up Cross-Cloud Interconnect (Optional): Configure a private connection between your Google Cloud VPC and your remote cloud provider.
  • Set up federation: Configure authentication and create a federated catalog in Lakehouse.
    • Secret-based authentication (PAT): Create a secret in Secret Manager with your remote catalog credentials. Then, create a federated catalog in Lakehouse and grant the catalog service account access to the secret.
    • Workload Identity Federation (WIF): Create a federated catalog in Lakehouse specifying the required Snowflake role. Then, link the catalog's service account ID to a service user in Snowflake. The Snowflake service user should have usage permissions on the remote Snowflake catalog.
  • Verify the connection: Verify that Lakehouse can successfully connect to your remote catalog.
  • Query data: Run queries against your federated data using BigQuery or Managed Service for Apache Spark. For more information, see Use cross-cloud Lakehouse.
  • Configure permissions: Use IAM to manage who can view and query the federated data.

Set up Cross-Cloud Interconnect (Optional)

Queries to your remote catalog travel over the public internet by default. To help enhance security and compliance, provide predictable performance, and reduce data transfer costs, use a private interconnect. This establishes a dedicated, private network connection between your Google Cloud Virtual Private Cloud (VPC) and your remote cloud provider's network (for example, AWS).

You can provision and configure either of the following private interconnect options between your Google Cloud VPC and your remote cloud provider's VPC (for example, AWS):

Establish BGP sessions between your Cloud Router in Google Cloud and your remote cloud provider's VPC to ensure route exchange.

To enable private querying, you must configure a path from Lakehouse to your remote storage bucket (for example, an AWS Amazon S3 bucket) through your private interconnect. There are two architectural flows you can follow to configure this routing:

  • Internal regional proxy Network Load Balancer routing: This flow uses a Google Cloud internal regional proxy Network Load Balancer to distribute requests across Hybrid Connectivity Network Endpoint Groups (NEGs) pointing to multiple AWS Elastic Network Interfaces (ENIs). This flow is essential for load balancing, scalability, and high availability. It is required for Partner CCI and recommended for Dedicated CCI for load balancing, scalability, and high availability.
  • Direct endpoint routing: This flow connects Service Directory directly to a single AWS Interface VPC Endpoint IP address. This flow only works for Dedicated CCI and is not supported for Partner CCI.

Select the configuration flow that matches your architecture requirements:

Internal regional proxy Network Load Balancer

To configure an internal regional proxy Network Load Balancer to distribute requests across multiple AWS ENIs for high availability and load balancing, follow these steps:

Configure AWS networking

First, create an Amazon S3 VPC Interface Endpoint (AWS PrivateLink):

  1. In the AWS VPC console, create an Interface Endpoint for Amazon S3.
  2. For the service name, specify com.amazonaws.AWS_REGION.s3.
  3. Select the VPC and subnets that are connected through Direct Connect to your Google Cloud VPC.
  4. Attach Security Groups to the endpoint to control inbound access.
  5. This provisions Elastic Network Interfaces (ENIs) in each selected subnet. Note the private IP addresses of these ENIs.

Next, configure Security Groups:

  • Ensure that the Security Group or groups attached to the Amazon S3 Endpoint ENIs allow inbound TCP traffic on port 443 from your Google Cloud VPC. This must include the CIDR range of your Google Cloud proxy-only subnet to allow health checks and forwarded traffic.

Configure Google Cloud networking

To simplify setup, run the following commands to configure the internal load balancer. For advanced configurations or more details, see Set up an Internal regional proxy Network Load Balancer for hybrid endpoints.

gcloud compute networks subnets create PROXY_SUBNET_NAME \
    --purpose=REGIONAL_MANAGED_PROXY \
    --role=ACTIVE \
    --region=REGION \
    --network=VPC_NETWORK \
    --range=PROXY_SUBNET_RANGE

Replace the following:

  • PROXY_SUBNET_NAME: a name for the proxy-only subnet.
  • PROXY_SUBNET_RANGE: an unused CIDR range within your VPC network (for example, 10.129.0.0/23).
  1. Create a regional health check:

    gcloud compute health-checks create tcp HEALTH_CHECK_NAME \
        --region=REGION \
        --port=443

    Replace the following:

    • HEALTH_CHECK_NAME: a name for the health check.
    • REGION: the Google Cloud region (for example, us-east4).
  2. Create hybrid Connectivity Network Endpoint Groups (NEGs) and add endpoints:

    Create a hybrid NEG (NON_GCP_PRIVATE_IP_PORT) for each zone:

    gcloud compute network-endpoint-groups create NEG_NAME \
        --network-endpoint-type=NON_GCP_PRIVATE_IP_PORT \
        --zone=ZONE \
        --network=VPC_NETWORK

    Add the private IP address of your AWS ENI to the corresponding hybrid NEG:

    gcloud compute network-endpoint-groups update NEG_NAME \
        --zone=ZONE \
        --add-endpoint="ip=AWS_S3_IP,port=443"

    Replace the following:

    • NEG_NAME: a name for the hybrid NEG.
    • ZONE: the Google Cloud zone (for example, us-east4-a). This zone must reside within the region of your Cross-Cloud Interconnect VLAN attachment.
    • VPC_NETWORK: the name of your VPC network.
    • AWS_S3_IP: the private IP address of the AWS Amazon S3 VPC Endpoint (ENI) in that zone.

    Repeat these commands to create NEGs and add endpoints for other zones if your AWS ENIs are distributed across multiple zones.

  3. Create and configure the backend service:

    Create a regional backend service with internal managed load balancing:

    gcloud compute backend-services create BACKEND_SERVICE_NAME \
        --load-balancing-scheme=INTERNAL_MANAGED \
        --protocol=TCP \
        --region=REGION \
        --health-checks=HEALTH_CHECK_NAME \
        --health-checks-region=REGION

    Add your hybrid NEGs to the backend service:

    gcloud compute backend-services add-backend BACKEND_SERVICE_NAME \
        --region=REGION \
        --network-endpoint-group=NEG_NAME \
        --network-endpoint-group-zone=ZONE \
        --balancing-mode=CONNECTION \
        --max-connections=MAX_CONNECTIONS

    Replace the following:

    • BACKEND_SERVICE_NAME: a name for the backend service.
    • NEG_NAME: the name of the hybrid NEG you created in the previous step.
    • ZONE: the Google Cloud zone (for example, us-east4-a).
    • MAX_CONNECTIONS: the maximum concurrent connections the backend should handle (for example, 100).

    Repeat the add-backend command for each hybrid NEG you created.

  4. Configure the load balancer frontend:

    Create a target TCP proxy:

    gcloud compute target-tcp-proxies create TARGET_PROXY_NAME \
        --backend-service=BACKEND_SERVICE_NAME \
        --region=REGION

    Create a forwarding rule to route traffic to the target proxy:

    gcloud compute forwarding-rules create FORWARDING_RULE_NAME \
        --load-balancing-scheme=INTERNAL_MANAGED \
        --network=VPC_NETWORK \
        --subnet=VPC_SUBNET \
        --ports=443 \
        --region=REGION \
        --target-tcp-proxy=TARGET_PROXY_NAME \
        --target-tcp-proxy-region=REGION \
        --allow-global-access

    Replace the following:

    • TARGET_PROXY_NAME: a name for the target proxy.
    • FORWARDING_RULE_NAME: a name for the forwarding rule.
    • VPC_SUBNET: the name of your VPC subnetwork.

After creating the forwarding rule for the load balancer, note the internal IP address assigned to it. This is your ILB_IP_ADDRESS.

Configure Service Directory

Register the ILB's IP address in Service Directory, so Lakehouse can discover it.

  1. Create a namespace for your remote cloud:

    gcloud service-directory namespaces create NAMESPACE \
        --project=PROJECT_ID \
        --location=REGION

    Replace the following:

    • NAMESPACE: a unique identifier for your namespace.
    • PROJECT_ID: your Google Cloud project ID.
    • REGION: the Google Cloud region. For example, us-east4. This must be the same region as the federated catalog.
  2. Create a service in the Service Directory namespace:

    gcloud service-directory services create SERVICE_NAME \
        --namespace=NAMESPACE \
        --project=PROJECT_ID \
        --location=REGION

    Replace the following:

    • SERVICE_NAME: a unique identifier for your service.
  3. Create an endpoint for the ILB in the service:

    gcloud service-directory endpoints create ENDPOINT_NAME \
        --project=PROJECT_ID \
        --namespace=NAMESPACE \
        --service=SERVICE_NAME \
        --location=REGION \
        --network=projects/PROJECT_NUMBER/global/networks/VPC_NETWORK \
        --address=ILB_IP_ADDRESS \
        --port=443

    Replace the following:

    • ENDPOINT_NAME: a unique identifier for your endpoint.
    • PROJECT_NUMBER: your Google Cloud project number. Use your project number in the --network flag.
    • ILB_IP_ADDRESS: the internal IP address of your ILB forwarding rule.

Direct endpoint

To configure Service Directory to route traffic directly to a single AWS Interface VPC Endpoint IP address, follow these steps:

  1. Create an Interface VPC Endpoint for Amazon S3 inside your AWS VPC. Note the IP address and port of this endpoint.
  2. Create a namespace for your remote cloud:

    gcloud service-directory namespaces create NAMESPACE \
        --project=PROJECT_ID \
        --location=REGION

    Replace the following:

    • NAMESPACE: a unique identifier for your namespace.
    • PROJECT_ID: your Google Cloud project ID.
    • REGION: the Google Cloud region. For example, us-east4. This must be the same region as the federated catalog.
  3. Create a service in the Service Directory namespace:

    gcloud service-directory services create SERVICE_NAME \
        --namespace=NAMESPACE \
        --project=PROJECT_ID \
        --location=REGION

    Replace the following:

    • SERVICE_NAME: a unique identifier for your service.
  4. Create an endpoint in the service containing the routing information for your Amazon S3 Interface VPC Endpoint:

    gcloud service-directory endpoints create ENDPOINT_NAME \
        --service=SERVICE_NAME \
        --namespace=NAMESPACE \
        --project=PROJECT_ID \
        --location=REGION \
        --address=S3_VPCE_IP_ADDRESS \
        --port=S3_VPCE_PORT \
        --network=projects/PROJECT_NUMBER/global/networks/VPC_NETWORK

    Replace the following:

    • ENDPOINT_NAME: a unique identifier for your endpoint.
    • S3_VPCE_IP_ADDRESS: the IP address of your Amazon S3 Interface VPC Endpoint. For example, 10.0.1.45.
    • S3_VPCE_PORT: the port number of your Amazon S3 Interface VPC Endpoint. For example, 443.
    • PROJECT_NUMBER: your Google Cloud project number. Use your project number in the --network flag.
    • VPC_NETWORK: the Google Cloud VPC network name associated with your private interconnect.

Set up federation

To query your data, you must set up a Lakehouse federated catalog that connects to your remote Snowflake catalog. Select your authentication method:

Secret-based (PAT)

Configure authentication

Federation requires credentials to access the remote Snowflake catalog. For Snowflake Horizon, you must use a Personal Access Token (PAT), which is a long-lived token generated by Snowflake, alongside the specific Snowflake role required for the session.

Create a secret in regional Secret Manager to store the credentials:

  1. Create a JSON file named credentials.json with your payload:

      {
        "client_secret": "SNOWFLAKE_PAT_TOKEN",
        "scope": "session:role:SNOWFLAKE_ROLE"
      }
      

    Replace the following:

    • SNOWFLAKE_PAT_TOKEN: Your Snowflake Personal Access Token (PAT).
    • SNOWFLAKE_ROLE: The specific Snowflake role required for the session. For example, ICEBERG_VIEW.
  2. Configure the regional endpoint for Secret Manager:

    By default, Secret Manager uses a global endpoint. However, cross-cloud Lakehouse requires that your secrets be stored in the same region as your Lakehouse catalog. To interact with regional secrets using the gcloud CLI, you must override the default API endpoint for your current session or profile. To avoid connectivity issues, your secret and your catalog must be created in the same region.

      gcloud config set api_endpoint_overrides/secretmanager https://secretmanager.REGION.rep.googleapis.com/
      

    Replace the following:

    • REGION: The Google Cloud region where your Secret Manager secret is stored. For example, us-east4. To avoid connectivity issues, your secret and your catalog must be created in the same region.
  3. Upload the payload to Secret Manager:

      gcloud secrets create SNOWFLAKE_SECRET_NAME \
        --location="REGION" \
        --project="PROJECT_ID" \
        --data-file=credentials.json
      

    Replace the following:

    • SNOWFLAKE_SECRET_NAME: A name for your Snowflake secret.
    • PROJECT_ID: your Google Cloud project ID.

    Create a federated catalog

    Create the federated catalog using the Google Cloud console or the gcloud CLI.

    Console

    To create a federated catalog using secret-based authentication:

    1. In the Google Cloud console, go to Lakehouse.

      Go to Lakehouse

    2. Click Create catalog.

    3. Click Federated catalog.

      The Catalog configuration details appear.

    4. For Federated catalog source, select Snowflake Horizon.

    5. For Data location, select the Lakehouse region where you want to create the federated catalog. For example, us-east4. To minimize latency (even over public internet) do the following when selecting a region:

      • If your Snowflake catalog is on AWS, select the Google Cloud region closest to your AWS region.
    6. Click Continue.

      The Connection details details appear.

    7. In the Remote catalog details section, in the Snowflake account identifier field, enter your Snowflake account identifier. For example: my_org-my_account.

    8. In the Snowflake warehouse field, enter your Snowflake warehouse name.

    9. For Secret, enter the name of your secret. Use the following format: projects/PROJECT_ID/locations/REGION/secrets/SNOWFLAKE_SECRET_NAME.

    10. Optional: In the Service directory name field, enter the path to your Service Directory endpoint or service. This is only required if you are configuring a private interconnect (Cross-Cloud Interconnect).

    11. Click Create.

    gcloud CLI

    Public internet (no CCI)

    If you don't configure CCI, the connection securely travels over the public internet.

      gcloud alpha biglake iceberg catalogs create FEDERATED_CATALOG_NAME \
          --project="PROJECT_ID" \
          --primary-location="REGION" \
          --catalog-type="federated" \
          --federated-catalog-type="snowflake" \
          --secret-name="projects/PROJECT_ID/locations/REGION/secrets/SNOWFLAKE_SECRET_NAME" \
          --snowflake-account-identifier="SNOWFLAKE_ACCOUNT_IDENTIFIER" \
          --snowflake-warehouse="SNOWFLAKE_WAREHOUSE" \
          --refresh-interval="REFRESH_INTERVAL" \
          --namespace-filters="NAMESPACE_FILTERS"
      

    Replace the following:

    • PROJECT_ID: your Google Cloud project ID.
    • REGION: the Lakehouse region where the federated catalog is created. For example, us-east4. To minimize latency, select the Google Cloud region closest to your Snowflake region.
    • SNOWFLAKE_SECRET_NAME: the name of your Snowflake secret.
    • SNOWFLAKE_ACCOUNT_IDENTIFIER: your Snowflake account identifier (for example, my_org-my_account).
    • SNOWFLAKE_WAREHOUSE: the Snowflake catalog name you want to federate with.
    • REFRESH_INTERVAL: Optional: Specifies how often to update the catalog's information. Set this value as a duration, for example, 330s or 5m30s. Shorter intervals update data more often but can cost more in API calls. Longer intervals can cost less, but the queried data might not reflect your most current dataset. If omitted or if the value is set to 0s, then background metadata refresh will not start. It will remain disabled until the refresh interval is updated to a positive value.
    • NAMESPACE_FILTERS: Optional: A comma-separated list of namespaces to federate. For example, ns1,ns2. If omitted, all namespaces will be included.

    Customer-owned (CCI)

    If you configured a private interconnect (such as Dedicated CCI or Partner CCI), provide the Service Directory endpoint reference so that Lakehouse routes traffic privately.

      gcloud alpha biglake iceberg catalogs create FEDERATED_CATALOG_NAME \
          --project="PROJECT_ID" \
          --primary-location="REGION" \
          --catalog-type="federated" \
          --federated-catalog-type="snowflake" \
          --secret-name="projects/PROJECT_ID/locations/REGION/secrets/SNOWFLAKE_SECRET_NAME" \
          --snowflake-account-identifier="SNOWFLAKE_ACCOUNT_IDENTIFIER" \
          --snowflake-warehouse="SNOWFLAKE_WAREHOUSE" \
          --refresh-interval="REFRESH_INTERVAL" \
          --namespace-filters="NAMESPACE_FILTERS" \
          --service-directory-name="projects/PROJECT_ID/locations/REGION/namespaces/NAMESPACE/services/SERVICE_NAME/endpoints/ENDPOINT_NAME"
      

    Replace the following:

    • PROJECT_ID: your Google Cloud project ID.
    • REGION: the Lakehouse region where the federated catalog is created. Note: This must be the same region as the Service Directory namespace and regional secret.
    • SNOWFLAKE_SECRET_NAME: the name of your Snowflake secret.
    • SNOWFLAKE_ACCOUNT_IDENTIFIER: your Snowflake account identifier.
    • SNOWFLAKE_WAREHOUSE: the Snowflake catalog name you want to federate with
    • REFRESH_INTERVAL: Optional: Specifies how often to update the catalog's information.
    • NAMESPACE_FILTERS: Optional: A comma-separated list of namespaces to federate.
    • NAMESPACE: the Service Directory namespace you created during private interconnect setup.
    • SERVICE_NAME: the Service Directory service name you created during private interconnect setup.
    • ENDPOINT_NAME: the Service Directory endpoint name you created during private interconnect setup.

    Complete authentication setup

    When the catalog is created, Lakehouse provisions a unique service account for it (returned as biglake-service-account in the resource description).

    You must grant this service account permission to access the secret you created earlier. Note that propagating IAM policies can take a few minutes.

    Grant the catalog's service account permission to access the secret:

    gcloud config set api_endpoint_overrides/secretmanager https://secretmanager.REGION.rep.googleapis.com/
    gcloud secrets add-iam-policy-binding SNOWFLAKE_SECRET_NAME \
      --project="PROJECT_ID" \
      --location="REGION" \
      --member="serviceAccount:$(gcloud alpha biglake iceberg catalogs describe FEDERATED_CATALOG_NAME \
          --project="PROJECT_ID" \
          --location="REGION" \
          --format='value(biglake-service-account)')" \
      --role="roles/secretmanager.secretAccessor"

    To verify that the federated catalog service account has access to the secret, run the following command:

    gcloud config set api_endpoint_overrides/secretmanager https://secretmanager.REGION.rep.googleapis.com/
    gcloud secrets get-iam-policy SNOWFLAKE_SECRET_NAME \
        --project="PROJECT_ID" \
        --location="REGION"

    In the output, verify that the biglake-service-account service account has the roles/secretmanager.secretAccessor role assigned to it.

Workload Identity Federation

Workload Identity Federation (WIF) avoids the use of long-lived secrets by linking a Lakehouse service account directly to a Snowflake service user.

Create a federated catalog

Create the federated catalog using the Google Cloud console, the Lakehouse REST API or the gcloud CLI. You must specify the Snowflake role to use.

Console

To create a federated catalog using Workload Identity Federation:

  1. In the Google Cloud console, go to Lakehouse.

    Go to Lakehouse

  2. Click Create catalog.

  3. Click Federated catalog.

    The Catalog configuration details appear.

  4. For Federated catalog source, select Snowflake Horizon.

  5. For Data location, select the Lakehouse region where you want to create the federated catalog. For example, us-east4. To minimize latency (even over public internet) do the following when selecting a region:

    • If your Snowflake catalog is on AWS, select the Google Cloud region closest to your AWS region.
  6. Click Continue.

    The Connection details details appear.

  7. In the Remote catalog details section, in the Snowflake account identifier field, enter your Snowflake account identifier. For example: my_org-my_account.

  8. In the Snowflake warehouse field, enter your Snowflake warehouse name.

  9. In the Snowflake role field, enter your Snowflake role. For example: ICEBERG_VIEW.

  10. Optional: In the Service directory name field, enter the path to your Service Directory endpoint or service. This is only required if you are configuring a private interconnect (Cross-Cloud Interconnect).

  11. Click Create.

REST API

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -H "x-goog-user-project: PROJECT_ID" \
  -d '{
    "catalog-type": "CATALOG_TYPE_FEDERATED",
    "federated-catalog-options": {
      "snowflake-catalog-info": {
        "account-identifier": "SNOWFLAKE_ACCOUNT_IDENTIFIER",
        "warehouse": "SNOWFLAKE_WAREHOUSE",
        "snowflake-role": "SNOWFLAKE_ROLE"
      },
      "refresh-options": {
        "refresh-schedule": {
          "refresh-interval": "REFRESH_INTERVAL"
        }
      }
    }
}' \
  "https://biglake.googleapis.com/iceberg/v1/restcatalog/extensions/projects/PROJECT_ID/catalogs?iceberg_catalog_id=FEDERATED_CATALOG_NAME&primary_location=REGION"

gcloud CLI

Public internet (no CCI)

    gcloud alpha biglake iceberg catalogs create FEDERATED_CATALOG_NAME \
        --project="PROJECT_ID" \
        --primary-location="REGION" \
        --catalog-type="federated" \
        --federated-catalog-type="snowflake" \
        --snowflake-account-identifier="SNOWFLAKE_ACCOUNT_IDENTIFIER" \
        --snowflake-warehouse="SNOWFLAKE_WAREHOUSE" \
        --snowflake-role="SNOWFLAKE_ROLE" \
        --refresh-interval="REFRESH_INTERVAL" \
        --namespace-filters="NAMESPACE_FILTERS"
    

Customer-owned (CCI)

If you configured a private interconnect, provide the Service Directory service reference so that Lakehouse routes traffic privately.

    gcloud alpha biglake iceberg catalogs create FEDERATED_CATALOG_NAME \
        --project="PROJECT_ID" \
        --primary-location="REGION" \
        --catalog-type="federated" \
        --federated-catalog-type="snowflake" \
        --snowflake-account-identifier="SNOWFLAKE_ACCOUNT_IDENTIFIER" \
        --snowflake-warehouse="SNOWFLAKE_WAREHOUSE" \
        --snowflake-role="SNOWFLAKE_ROLE" \
        --refresh-interval="REFRESH_INTERVAL" \
        --namespace-filters="NAMESPACE_FILTERS" \
        --service-directory-name="projects/PROJECT_ID/locations/REGION/namespaces/NAMESPACE/services/SERVICE_NAME/endpoints/ENDPOINT_NAME"
    

Replace the following:

  • PROJECT_ID: your Google Cloud project ID.
  • SNOWFLAKE_ACCOUNT_IDENTIFIER: your Snowflake account identifier.
  • SNOWFLAKE_WAREHOUSE: the Snowflake catalog name you want to federate with.
  • SNOWFLAKE_ROLE: the specific Snowflake role required for the session. For example, ICEBERG_VIEW.
  • FEDERATED_CATALOG_NAME: a name for the Lakehouse federated catalog.
  • REGION: the Lakehouse region where the federated catalog is created.
  • REFRESH_INTERVAL: Optional: Specifies how often to update the catalog's information. For example, 300s.
  • NAMESPACE_FILTERS: Optional: A comma-separated list of namespaces to federate. For example, ns1,ns2. If omitted, all namespaces will be included.
  • NAMESPACE: the namespace for your Service Directory service.
  • SERVICE_NAME: the name of your Service Directory service.
  • ENDPOINT_NAME: the name of your Service Directory endpoint.

After creating the catalog, you must link its service account identity to a service user in Snowflake.

  1. Extract the Lakehouse Service Account ID (Subject) from the catalog details.

    You can get this from the JSON response of the creation command (field biglake-service-account-id).

    Alternatively, you can run the describe command on the catalog to get the value:

    gcloud alpha biglake iceberg catalogs describe FEDERATED_CATALOG_NAME \
    --project="PROJECT_ID"

    Look for the biglake-service-account-id in the output.

  2. Log into your Snowflake management instance and execute the following script to establish the trust relationship with the Lakehouse service identity:

    USE ROLE ACCOUNTADMIN;
    
    CREATE USER SNOWFLAKE_SERVICE_USER
    TYPE = SERVICE
    WORKLOAD_IDENTITY = (
      TYPE = GCP
      SUBJECT = 'LAKEHOUSE_SERVICE_ACCOUNT_ID'
    )
    DEFAULT_ROLE = SNOWFLAKE_ROLE
    COMMENT = 'Service user for Lakehouse federation over WIF';
    
    -- Also explicitly GRANT permissions to the role
    GRANT ROLE SNOWFLAKE_ROLE TO USER SNOWFLAKE_SERVICE_USER;

    Replace the following:

    • SNOWFLAKE_SERVICE_USER: a name for the new service user in Snowflake.
    • LAKEHOUSE_SERVICE_ACCOUNT_ID: the service account ID extracted in the previous step.
    • SNOWFLAKE_ROLE: the Snowflake role (must match the role specified during catalog creation).

Verify the connection

Verify that the catalog background metadata refresh cycle completed successfully and namespaces are synchronized.

gcloud CLI

  1. Verify that the refresh status indicates success:

    gcloud alpha biglake iceberg catalogs describe FEDERATED_CATALOG_NAME \
      --project="PROJECT_ID" \
      --location="REGION"
  2. Confirm that remote database schemas appear as synchronized namespaces:

    gcloud alpha biglake iceberg namespaces list \
      --catalog="FEDERATED_CATALOG_NAME" \
      --project="PROJECT_ID" \
      --location="REGION"

REST API

  1. Verify catalog federation sync status:

    curl -X GET \
      -H "Authorization: Bearer $(gcloud auth print-access-token)" \
      -H "Content-Type: application/json" \
      -H "x-goog-user-project: PROJECT_ID" \
      "https://biglake.googleapis.com/iceberg/v1/restcatalog/extensions/projects/PROJECT_ID/catalogs/FEDERATED_CATALOG_NAME"
  2. List synchronized namespaces:

    curl -X GET \
      -H "Authorization: Bearer $(gcloud auth print-access-token)" \
      -H "Content-Type: application/json" \
      -H "x-goog-user-project: PROJECT_ID" \
      "https://biglake.googleapis.com/iceberg/v1/restcatalog/v1/projects/PROJECT_ID/catalogs/FEDERATED_CATALOG_NAME/namespaces"
  3. List tables in a synchronized namespace:

    curl -X GET \
      -H "Authorization: Bearer $(gcloud auth print-access-token)" \
      -H "Content-Type: application/json" \
      -H "x-goog-user-project: PROJECT_ID" \
      "https://biglake.googleapis.com/iceberg/v1/restcatalog/v1/projects/PROJECT_ID/catalogs/FEDERATED_CATALOG_NAME/namespaces/NAMESPACE_NAME/tables"

What's next