gcloud alpha bq migration validation run

NAME

gcloud alpha bq migration validation run - validate data migrated to BigQuery against its source

SYNOPSIS

gcloud alpha bq migration validation run CONFIG_PATH [CONFIG_PATH …] --location=LOCATION [--endpoint-mode=ENDPOINT_MODE] [--gcs-path=GCS_PATH] [--labels=[KEY=VALUE,…]] [--tables=[TABLE,…]] [--validation-types=[TYPE,…]] [GCLOUD_WIDE_FLAG …] [-- CLIENT_ARGS …]

DESCRIPTION

(ALPHA) Validate the data of the tables described by the configuration files generated by gcloud alpha bq migration validation configure. For each table, the BigQuery Migration Validation Client connects to the source data warehouse, extracts the schema, aggregates, and row samples of the table, stages them in the Cloud Storage location given by the configuration (or --gcs-path), and creates a BigQuery Migration Service validation workflow in --location that compares them with the migrated BigQuery table. The client prints the workflow name, the staged Cloud Storage path, and the BigQuery results location of each table, and exits when all workflows have been created.

The client runs on this machine and requires Java 11 or later on your PATH; the bqms component of the gcloud CLI, which contains the client, is installed on first use if it is missing. The client authenticates to Google Cloud with Application Default Credentials (for example, from gcloud auth application-default login), which must be allowed to write to the Cloud Storage staging location, to create BigQuery Migration Service workflows in the project of the migrated tables, and to run BigQuery jobs in that project that read the migrated tables and write the validation results. The credentials of the source data warehouse are read from the connections.yaml file of the configuration. To pass additional options to the Java virtual machine, set the JAVA_OPTS environment variable, for example export JAVA_OPTS=-Xmx8G (Linux and macOS) or set JAVA_OPTS=-Xmx8G (Windows Command Prompt).

Arguments after -- are forwarded to the validation client unchanged.

EXAMPLES

To validate the tables configured under ./validation_config with validation workflows in location eu, run:

gcloud alpha bq migration validation run validation_config --location=eu

To validate only the tables sales.orders and sales.customers, staging the source samples under gs://my-bucket/validation/, run:

gcloud alpha bq migration validation run validation_config --location=eu --tables=sales.orders,sales.customers --gcs-path=gs://my-bucket/validation/

To only compare the schemas and aggregates of the configured tables, run:

gcloud alpha bq migration validation run validation_config --location=eu --validation-types=schema,aggregate

POSITIONAL ARGUMENTS

CONFIG_PATH [CONFIG_PATH …]

Paths or glob patterns of the configuration files or directories generated by gcloud alpha bq migration validation configure.

[-- CLIENT_ARGS …]

Arguments forwarded to the validation client.

The '--' argument must be specified between gcloud specific args on the left and CLIENT_ARGS on the right.

REQUIRED FLAGS

--location=LOCATION

Location of the BigQuery Migration Service validation workflows (for example, eu or us-central1).

OPTIONAL FLAGS

--endpoint-mode=ENDPOINT_MODE

Specifies endpoint mode for a given command. Regional endpoints provide enhanced data residency and reliability by ensuring your request is handled entirely within the specified Google Cloud region. This differs from global endpoints, which may process parts of the request outside the target region. Overrides the default regional/endpoint_mode property value for this command invocation. ENDPOINT_MODE must be one of:

global

(Default) Use global rather than regional endpoints.

regional

Only use regional endpoints. An error will be raised if a regional endpoint is not available for a given command.

regional-preferred

Use regional endpoints when available, otherwise use global endpoints. Recommended for most users.

--gcs-path=GCS_PATH

Cloud Storage path (gs://bucket/path) where the source samples are staged. Overrides the path in the configuration.

--labels=[KEY=VALUE,…]

Labels to attach to the validation run.

--tables=[TABLE,…]

Only validate these tables. A table matches by its full source or target name or by a dot-separated suffix of either (for example, orders or sales.orders).

--validation-types=[TYPE,…]

Validations to run: schema compares the table schemas, aggregate compares column aggregates, and data compares sampled rows. Defaults to all of them. TYPE must be one of: schema, aggregate, data.

GCLOUD WIDE FLAGS

These flags are available to all commands: --access-token-file, --account, --billing-project, --configuration, --flags-file, --flatten, --format, --help, --impersonate-service-account, --log-http, --project, --quiet, --trace-token, --user-output-enabled, --verbosity.

Run $ gcloud help for details.

REGIONAL ENDPOINTS

This command supports regional endpoints. To use regional endpoints for this command, use the --endpoint-mode=regional-preferred flag. To use regional endpoints by default, run $ gcloud config set regional/endpoint_mode regional-preferred.

NOTES

This command is currently in alpha and might change without notice. If this command fails with API permission errors despite specifying the correct project, you might be trying to access an API with an invitation-only early access allowlist.