REST Resource: projects.locations.services.migrationExecutions

Resource: MigrationExecution

The details of a migration execution resource.

JSON representation 
{
  "name": string,
  "createTime": string,
  "endTime": string,
  "state": enum (State),
  "phase": enum (Phase),
  "stateMessage": string,

  // The following is a list of mutually exclusive fields. At most one of the
  // fields will be set in a response:
  "biglakeMetastoreMigrationConfig": {
    object (BigLakeMetastoreMigrationConfig)
  }
  // End of mutually exclusive fields.
}
Fields
name

string

Output only. The relative resource name of the migration execution, in the following form: projects/{projectNumber}/locations/{locationId}/services/{serviceId}/migrationExecutions/{migrationExecutionId}
createTime

string (Timestamp format)

Output only. The time when the migration execution was started.

Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z", "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30".
endTime

string (Timestamp format)

Output only. The time when the migration execution finished.

Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z", "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30".
state

enum (State)

Output only. The current state of the migration execution.
phase
(deprecated)

enum (Phase)

Output only. Deprecated: Phase was designed for incoming migrations to Dataproc Metastore, not applicable when migrating away from it. The current phase of the migration execution.
stateMessage

string

Output only. Additional information about the current state of the migration execution.
Configuration information required by Dataproc Metastore for starting the migration process The following is a list of mutually exclusive fields. At most one of the fields will be set in a response:
biglakeMetastoreMigrationConfig

object (BigLakeMetastoreMigrationConfig)

Configuration information specific to migrating from Dataproc Metastore to BigLake Metastore.
End of mutually exclusive fields.

BigLakeMetastoreMigrationConfig

Defines the configuration required to migrate metadata from a Dataproc Metastore service to BigLake Metastore.

JSON representation 
{
  "mode": enum (MigrationMode),
  "dryRun": boolean,
  "reportPath": string,
  "hiveConfig": {
    object (HiveConfig)
  },
  "icebergConfig": {
    object (IcebergConfig)
  },
  "backfillStatus": {
    object (BackfillStatus)
  },
  "conflictPolicy": enum (ConflictPolicy)
}
Fields
mode

enum (MigrationMode)

Required. Defines the behavior of the migration execution.
dryRun

boolean

Optional. If true, performs discovery of requested resources and analysis against the target catalog to come up with a plan for each resource (e.g. Create, Update, Skip, etc.). No metadata is actually migrated.
reportPath

string

Optional. The Cloud Storage path where the backfill / dry run report should be written. If not provided, the report will be generated in the service's artifacts bucket. Format: "gs://path/to/folder"
hiveConfig

object (HiveConfig)

Optional. At least one of hiveConfig or icebergConfig must be provided, otherwise, a validation error will be thrown. If only one is provided, the service only migrates tables of that specific type. If both are provided, both Hive and Iceberg tables will be migrated.

Configuration for migrating Hive tables to a BigLake Hive catalog.
icebergConfig

object (IcebergConfig)

Optional. Configuration for migrating Iceberg tables to a BigLake Iceberg REST catalog.
backfillStatus

object (BackfillStatus)

Output only.

conflictPolicy

enum (ConflictPolicy)

Optional. The policy to handle conflicts when migrating resources, defaults to SKIP if not specified.

MigrationMode

The execution mode of the migration.

Enums
MIGRATION_MODE_UNSPECIFIED The migration mode is unspecified.
BACKFILL Performs the metadata migration of requested resources. The migration completes once the backfill is finished.
INCREMENTAL_SYNC Performs the initial backfill, then transitions to an incremental synchronization state to replicate ongoing source writes. Requires an explicit CompleteMigration or CancelMigration call to end the migration. Note: Supported only for Iceberg migrations.

HiveConfig

Configuration for migrating Hive metadata.

JSON representation 
{
  "catalog": string,
  "databases": [
    string
  ]
}
Fields
catalog

string

Required. The target catalog for migrated databases and tables. Format: "projects/{project_id_or_number}/catalogs/{catalog_id}"
databases[]

string

Required. The list of databases to migrate to the Hive catalog. Use "*" to migrate all databases. Note: If Iceberg tables exist in these databases, they will only be migrated if icebergConfig is also specified.

IcebergConfig

Configuration for migrating Iceberg metadata.

JSON representation 
{
  "catalog": string,
  "namespaces": [
    string
  ]
}
Fields
catalog

string

Required. The target catalog for migrated Iceberg metadata. Format: "projects/{project_id_or_number}/catalogs/{catalog_id}"
namespaces[]

string

Required. The list of namespaces to migrate to the Iceberg REST catalog. Use "*" to migrate all namespaces. Note: If Hive tables exist in these namespaces, they will only be migrated if hiveConfig is also specified.

BackfillStatus

Backfill status for the migration execution.

JSON representation 
{
  "state": enum (State),
  "reportPath": string,
  "migrationSummary": {
    object (MigrationSummary)
  }
}
Fields
state

enum (State)

Output only. The current state of the backfill (or dry run).
reportPath

string

Output only. The Cloud Storage path where the backfill or dry run report is written. Format: "gs://path-to-report".
migrationSummary

object (MigrationSummary)

Output only. Summary of the migration results. This is populated after the backfill or dry run is finished.

State

The backfill state.

Enums
STATE_UNSPECIFIED The backfill state is unspecified.
PENDING Waiting to start.
RUNNING Backfill in progress.
SUCCEEDED Backfill complete, report is available
FAILED Backfill failed; check report for details

MigrationSummary

Summary of the migration results.

JSON representation 
{
  "createTime": string,
  "service": string,
  "dryRun": boolean,
  "catalogSummaries": [
    {
      object (CatalogSummary)
    }
  ]
}
Fields
createTime

string (Timestamp format)

Output only. The UTC time when this report was finalized.

Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z", "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30".
service

string

Output only. The Dataproc Metastore service name (format: projects/*/locations/*/services/*) on which the migration was executed.
dryRun

boolean

Output only. Whether the migration was a dry run.
catalogSummaries[]

object (CatalogSummary)

Output only. Summary of results for each catalog involved in the migration.

CatalogSummary

Summary of results for a specific destination catalog.

JSON representation 
{
  "catalog": string,
  "catalogType": enum (CatalogType),
  "databaseSummaries": [
    {
      object (DatabaseSummary)
    }
  ]
}
Fields
catalog

string

Output only. The catalog resource name (format: projects/*/catalogs/*).
catalogType

enum (CatalogType)

Output only. The type of the catalog.
databaseSummaries[]

object (DatabaseSummary)

Output only. Summary of results for each database in the catalog.

CatalogType

The type of catalog.

Enums
CATALOG_TYPE_UNSPECIFIED The catalog type is unspecified.
HIVE BigLake Metastore Hive catalog.
ICEBERG BigLake Metastore Iceberg REST catalog.

DatabaseSummary

Summary of results for a specific database in a catalog.

JSON representation 
{
  "database": string,
  "planAction": enum (Action),
  "resultStatus": enum (State),
  "tableSummary": {
    object (TableSummary)
  }
}
Fields
database

string

Output only. The name of the database.
planAction

enum (Action)

Output only. The migration plan action for the database.
resultStatus

enum (State)

Output only. The migration result status for the database. This is only set if the migration is not a dry run.
tableSummary

object (TableSummary)

Output only. Aggregated summary of results for all tables in the database.

Action

The action that will be taken for a resource during migration.

Enums
ACTION_UNSPECIFIED The action is unspecified.
CREATE Resource missing; will be created.
UPDATE Resource exists at the target, but differs from the source; will be updated.
SKIP Resource exists at the target; no changes will be made.
DEPENDENCY_FAILURE Resource cannot be migrated due to a dependency failure (e.g., parent resource missing).
ERROR Resource cannot be migrated due to an error during discovery.

State

The state of the migration for a resource.

Enums
STATE_UNSPECIFIED The state is unspecified.
SUCCEEDED The resource was migrated successfully.
FAILED The resource failed to migrate.
SKIPPED The resource was skipped and will not be migrated.

TableSummary

Aggregated summary of results for all tables in a database.

JSON representation 
{
  "planCounts": {
    string: string,
    ...
  },
  "resultCounts": {
    string: string,
    ...
  },
  "partitionDiscoveredCount": string,
  "partitionSuccessCount": string,
  "partitionFailedCount": string
}
Fields
planCounts

map (key: string, value: string (int64 format))

Output only. Number of tables with a specific migration plan action. The key is the action name (e.g. CREATE, UPDATE, SKIP, etc.).

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
resultCounts

map (key: string, value: string (int64 format))

Output only. Number of tables with a specific migration result status. The key is the status name (e.g. SUCCEEDED, FAILED, SKIPPED, etc.). This is only set if the migration is not a dry run.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
partitionDiscoveredCount

string (int64 format)

Output only. Partition migration summary across all Hive tables in the database.

The total number of partitions discovered at the source.
partitionSuccessCount

string (int64 format)

Output only. The total number of partitions successfully migrated at the target.
partitionFailedCount

string (int64 format)

Output only. The total number of partitions that failed to migrate at the target.

ConflictPolicy

The policy to handle conflicts when migrating resources to Lakehouse.

Enums
CONFLICT_POLICY_UNSPECIFIED The conflict policy is unspecified.
SKIP Skip migrating resources that already exist in the target catalog.
OVERWRITE Update resources that already exist in the target catalog.

State

The current state of the migration execution.

Enums
STATE_UNSPECIFIED The state of the migration execution is unknown.
STARTING The migration execution is starting.
RUNNING The migration execution is running.
CANCELLING The migration execution is in the process of being cancelled.
AWAITING_USER_ACTION The migration execution is awaiting user action.
SUCCEEDED The migration execution has completed successfully.
FAILED The migration execution has failed.
CANCELLED The migration execution is cancelled.
DELETING The migration execution is being deleted.
ROLLED_BACK The migration execution has been rolled back. This occurs when CancelMigration is invoked after the migration has completed and the metastore service is in the PROXY state, returning the service to the ACTIVE state. This enables rollback support when the customer wants to resume using DPMS after a successful migration.

Phase

Deprecated: Phase was designed for incoming migrations to Dataproc Metastore, not applicable when migrating away from it. The current phase of the migration execution.

Enums
PHASE_UNSPECIFIED The phase of the migration execution is unknown.
REPLICATION Replication phase refers to the migration phase when Dataproc Metastore is running a pipeline to replicate changes in the customer database to its backend database. During this phase, Dataproc Metastore uses the customer database as the hive metastore backend database.
CUTOVER Cutover phase refers to the migration phase when Dataproc Metastore switches to using its own backend database. Migration enters this phase when customer is done migrating all their clusters/workloads to Dataproc Metastore and triggers CompleteMigration.

Methods

delete

 Deletes a single migration execution.

get

 Gets details of a single migration execution.

list

 Lists migration executions on a service.