Post-deployment steps

This page guides you through the post-deployment steps of Cortex Framework to execute your data pipelines and verify the results.

After you run your deployment, all transformation templates and configurations are compiled and deployed to your target Dataform repository. However, to materialize the tables and views in BigQuery, you must first execute the Dataform pipeline actions.

Execute the transformations in Dataform

By executing the pipeline, you initiate the logic transformation of raw data into standardized, AI-ready data foundation layers and Data Products, ready for immediate business use.

You can execute actions manually in the Google Cloud console for verification, or configure recurring schedules for production.

Manual execution for verification

To execute a manual verification run of the entire pipeline within your Dataform workspace, follow these steps:

  1. In the Google Cloud console, go to the Dataform page.
  2. Confirm that you have selected the correct Google Cloud project from the project drop-down.
  3. Click the name of the repository created during deployment (e.g., cortex).
  4. Click the workspace name (e.g., cortex).
  5. Click Start execution.
  6. Click Execute actions.
  7. Select All actions. You can also choose key subsets of transformations by selecting specific tags or actions.
  8. Click Start execution.

Verify execution success

To monitor progress and check whether the pipeline completed successfully, follow these steps:

  1. Click the Executions tab to view the execution history.
  2. Verify that the status is Completed successfully.
  3. Click the execution run link to inspect details of specific actions, compilation errors, or execution timings.

Review data assets in BigQuery

After your Dataform pipeline executes successfully, verify the generated datasets and look at the materialized tables in BigQuery.

  1. Navigate to BigQuery in the Google Cloud console.
  2. Locate the target datasets configured in your config.yaml file (e.g., the dataset identified by datasetId in the data.targets entry referenced by the module's dataTargetId). For example, in the template configurations, these may be:
    • Data foundation layer: Look for the dataset matching your foundation target ID (configured as cortex7_sap_data_foundation by default).
    • Data products layer: Look for the dataset matching your product target ID (configured as cortex7_data_products by default).
  3. Preview the tables and run sample queries to verify that they are populated with the expected records.

Next steps

Now that your data systems are operational, you can explore: