Set up cross-cloud Lakehouse for SAP Business Data Cloud

This document describes how to set up a cross-cloud Lakehouse to query data in SAP Business Data Cloud (BDC) directly within Google Cloud. This integrates external data sources with your existing Google Cloud environment.

For an overview of how cross-cloud Lakehouse integrates with SAP BDC, see About SAP BDC integration.

Before you begin

  1. Review the Lakehouse overview to understand how Lakehouse manages data access.
  2. Read About cross-cloud Lakehouse to understand how it works.
  3. Review the supported catalogs to verify requirements.
  4. Ensure your SAP administrator has access to your SAP BDC environment to configure the BDC Connect application.
  5. Ensure your target Google Cloud project runs on BigQuery Enterprise Plus edition.
  6. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

  7. Verify that billing is enabled for your Google Cloud project.

  8. Enable the BigLake API.

    Roles required to enable APIs

    To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

    Enable the API

  9. Verify that billing is enabled for your Google Cloud project.

  10. Enable the BigLake API.

    Roles required to enable APIs

    To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

    Enable the API

    11.

Required roles

To get the permissions that you need to set up cross-cloud Lakehouse, ask your administrator to grant you the following IAM roles on your project:

  • Manage Lakehouse catalogs: BigLake Admin (roles/biglake.admin) or BigQuery Admin (roles/bigquery.admin)

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Limitations and considerations

This section lists the limitations and considerations for using cross-cloud Lakehouse.

  • SAP delta sharing tables don't support fine-grained access control (FGAC).
  • Specific VPC configuration or allowlisting might be required depending on your network settings. For VPC Service Controls configurations, see VPC Service Controls configuration for SAP BDC.
  • Data Products with special characters like / or - are not supported and will not be available in BigQuery even if shared from SAP BDC to BigQuery. If you share a Data Product with special characters, this could cause the refresh to break on the Google side and require re-enrollment. Known SAP systems producing these Data Products include BW sources and SuccessFactors.
  • Data freshness: The --refresh-interval flag for the federated catalog determines how often metadata is synchronized. A shorter interval provides fresher data but can incur additional API costs from the remote catalog provider.
  • Iceberg Metrics Reporting: Iceberg Metrics Reporting is not available for federated catalogs. Set the rest-metrics-reporting-enabled property to false in your Iceberg client when accessing a federated catalog.

General workflow

To set up and use cross-cloud Lakehouse with SAP BDC, follow these steps:

  • Set up Enterprise Plus edition: Enable BigQuery Enterprise Plus when querying your SAP BDC instance.
  • Create the Lakehouse catalog: A BigQuery Administrator creates an empty Delta Sharing catalog to connect to SAP BDC.
  • Configure the connector in SAP for Me: Your SAP administrator provisions a BDC Connect application instance.
  • Update the catalog: Complete the handshake by adding the endpoint and invitation code to the Lakehouse catalog.
  • Share SAP BDC data products: Your SAP data steward shares data products for BigQuery.
  • Query data: Query your federated data using BigQuery.
  • Configure permissions: Use Identity and Access Management (IAM) to manage access.

Set up BigQuery Enterprise Plus edition

This one-time setup lets BigQuery use Enterprise Plus to query your SAP BDC instance.

Create an Enterprise Plus reservation

  1. Go to BigQuery.

    Go to BigQuery

  2. In the navigation menu, go to Administration > Capacity management.

  3. For Reservation Name, enter a name (for example, enterprise-plus-sap-res).

  4. Click the Reservations tab and then click Create Reservation.

  5. For Location, select the region where your data resides. This must match the region used for the Lakehouse catalog.

  6. For Edition, select Enterprise Plus.

  7. For Slots, set the following:

    • Set Baseline slots to 0 to prevent charges while not querying data.
    • Enable Autoscaling to let BigQuery scale slots as needed.

  8. Click Save.

Assign your project to the reservation

  1. In the Capacity management page, click the Assignments tab.
  2. Click Create Assignment.
  3. For Select a resource, choose the project, folder, or organization that runs the BigQuery jobs.
  4. For Reservation, select the Enterprise Plus reservation you created.
  5. For Job type, select Query.
  6. Click Create.

To verify, check the Job Information in the BigQuery Job History for any job run within that project.

Set up cross-cloud federation

To query your data, set up a Lakehouse federated catalog that connects to your remote SAP BDC catalog. This initiates a handshake that establishes a trust relationship where the BigQuery service account acts as the identity for the Google Cloud project.

Create the Lakehouse Delta Sharing catalog

Create an empty Delta Sharing catalog using the Google Cloud console or the gcloud CLI.

Console

  1. In the Google Cloud console, go to the Lakehouse page.

    Go to Lakehouse

  2. In the navigation menu, click Runtime catalog.

  3. Click Create catalog and select Federated catalog.

  4. In the Catalog configuration section, enter the following:

    • For Federated catalog source, select SAP (Delta).
    • For Catalog name (in Lakehouse), enter a name for your catalog.
    • For Data location, select the Lakehouse region.

  5. Click Continue.

  6. Retrieve the Google Cloud Service Account ID displayed on the page and provide this ID to your SAP administrator.

gcloud

  1. Create an empty Delta Sharing catalog:

    gcloud alpha biglake delta-sharing catalogs create CATALOG_ID \
    --project="PROJECT_ID" \
    --location="REGION" \
    --refresh-interval="REFRESH_INTERVAL"

    Replace the following:

    • CATALOG_ID: a name for your catalog.
    • PROJECT_ID: your Google Cloud project ID.
    • REGION: the Lakehouse region.
    • REFRESH_INTERVAL: Optional: the interval for refreshing metadata from the catalog, specified as a duration string with units (for example, 5m for 5 minutes, 300s for 300 seconds, or 1h for 1 hour). If omitted, the refresh interval defaults to 5 minutes (5m or 300s).

  2. Retrieve the Google Cloud Service Account ID to provide to your SAP administrator:

    gcloud alpha biglake delta-sharing catalogs describe CATALOG_ID \
    --project="PROJECT_ID"

Configure the connector in SAP for Me

Your SAP administrator must sign in to SAP for Me and provision a new instance of the BDC Connect application:

  1. Enter the Display Name.
  2. Select or create a Resource Group.
  3. Allocate BDC Capacity Units.
  4. Enter the Google Cloud service account ID as the External System Instance Identifier.
  5. Select a Region. This must match the BDC Core region.
  6. Click Finish.

After completion, go to the tenant details view. Hold the pointer over the View Tenant Notifications button to get the Activation Link, which contains the SAP Connector Endpoint and the SAP Invitation Code.

Provide these to your BigQuery administrator.

For more information, see Provisioning SAP Business Data Cloud Connect in the SAP documentation.

Update the catalog with SAP enrollment details

Update the catalog to complete the handshake using the Google Cloud console or the gcloud CLI.

Console

  1. If you are not already on the Connection details page:

    1. In the Google Cloud console, go to the Lakehouse page.

      Go to Lakehouse

    2. In the navigation menu, click Runtime catalog.

    3. Select your SAP Delta Sharing catalog to open its Catalog details page.

    4. In the banner, click Enroll catalog.

  2. In the Connection details section, enter the following:

    • For Connector endpoint, enter the URL provided by your SAP administrator.
    • For Invitation code, enter the code provided by your SAP administrator.

  3. Click Submit.

gcloud 

gcloud alpha biglake delta-sharing catalogs update CATALOG_ID \
    --project="PROJECT_ID" \
    --connector-endpoint="CONNECTOR_ENDPOINT" \
    --invitation-code="INVITATION_CODE" \
    --refresh-interval="REFRESH_INTERVAL"

Replace the following:

  • CATALOG_ID: the name of your catalog.
  • PROJECT_ID: your Google Cloud project ID.
  • CONNECTOR_ENDPOINT: the URL provided by the SAP administrator.
  • INVITATION_CODE: the code provided by the SAP administrator.
  • REFRESH_INTERVAL: Optional: the interval for refreshing metadata from the catalog, specified as a duration string with units (for example, 5m for 5 minutes, 300s for 300 seconds, or 1h for 1 hour). If omitted, the background metadata refresh interval defaults to 5 minutes (5m or 300s).

Share SAP BDC Data Products

Your SAP data steward must sign in to SAP Business Data Cloud and share data products for BigQuery:

  1. Browse the Catalog and select a shareable Data Product.
  2. Go to the API.
  3. Click Share.
  4. Click Add Target and select the BDC Connect instance for Google BigQuery.
  5. Click Share.

The shared Data Product is discoverable in Google BigQuery. For more information, see Working with Data Products in SAP Business Data Cloud Connect in the SAP documentation.

What's next