Notebooks and programmatic analysis tools

While SQL is a powerful query language, programming languages like Python, Java, or R provide syntaxes and statistical functions that can be more expressive for data analysis tasks. Notebook environments offer a highly flexible alternative to spreadsheets for complex data exploration and machine learning (ML) workflows.

BigQuery provides built-in notebook integration as well as support for several other programmatic analysis solutions.

Colab Enterprise notebooks

You can use Colab Enterprise notebooks in BigQuery to perform end-to-end data science and ML workflows in a single, integrated interface. Unlike standard SQL editors, notebooks let you combine SQL queries with Python code, rich text, and visualizations to tell a comprehensive story with your data. Colab Enterprise notebooks are ideal for the following use cases:

End-to-end ML workflows: build, evaluate, and deploy BigQuery ML models within a single interface.
Data exploration: clean and analyze large datasets using BigQuery DataFrames.
Collaborative research: track version history and share notebooks with colleagues using Identity and Access Management (IAM).

Colab Enterprise notebooks offer the following benefits:

AI-powered development: leverage Gemini Enterprise Agent Platform for assistive code development.
Seamless Python integration: use the BigQuery DataFrames API without any additional setup.
Familiar editor features: benefit from SQL auto-completion similar to the BigQuery SQL editor.
Integrated visualizations: use interactive DataFrame visualizations or libraries like matplotlib and seaborn directly in your workflow.
SQL-Python interoperability: execute SQL in cells that reference Python variables.

Notebook gallery

The notebook gallery is a central hub for discovering and using prebuilt notebook templates. It includes fundamental templates for SQL, Python, Apache Spark, and DataFrames to help you perform common tasks like data preparation, analysis, and visualization.

Runtime management and security

Notebooks use Colab Enterprise runtimes, which are Compute Engine virtual machines allocated to a specific user to enable code execution. Access to notebooks is controlled using IAM roles. To detect vulnerabilities in the Python packages used in your notebooks, you can install and use the Notebook Security Scanner.

Monitoring and regions

All code assets in BigQuery Studio are stored in a default region, and updating this setting changes the region for any newly created code assets. Colab Enterprise notebooks are available in all regions where BigQuery Studio is available.

To monitor notebook slot usage, you can view your Cloud Billing report and apply a filter for the goog-bq-feature-type label with the value BQ_STUDIO_NOTEBOOK.

Notebook capabilities are available only in the Google Cloud console.

Pricing

For pricing information about Colab Enterprise notebooks, see Notebook runtime pricing.

BigQuery DataFrames

BigQuery DataFrames is a set of open source Python libraries that let you take advantage of BigQuery data processing by using familiar Python APIs. BigQuery DataFrames implements the pandas and scikit-learn APIs by pushing the processing down to BigQuery through SQL conversion. This design lets you use BigQuery to explore and process terabytes of data and train ML models, all with Python APIs.

BigQuery DataFrames offers the following benefits:

More than 750 pandas and scikit-learn APIs implemented through transparent SQL conversion to BigQuery and BigQuery ML APIs.
Deferred execution of queries for enhanced performance.
Extending data transformations with user-defined Python functions to let you process data in the cloud. These functions are automatically deployed as BigQuery remote functions.
Integration with Gemini Enterprise Agent Platform to let you use Gemini models for text generation.