# JDBC to Cloud Spanner template

Use the Managed Service for Apache Spark JDBC to Spanner template to extract
data from JDBC databases to Spanner.

This template supports the following databases as input:

- MySQL
- PostgreSQL
- Microsoft SQL Server
- Oracle

## Use the template

Run the template using the gcloud CLI or Managed Service for Apache Spark
API.

### gcloud


Before using any of the command data below,
make the following replacements:

- <var class="edit" scope="PROJECT_ID" translate="no">PROJECT_ID</var>: Required. Your Google Cloud project ID listed in the [IAM Settings](https://console.cloud.google.com/iam-admin/settings/project).
- <var class="edit" scope="REGION" translate="no">REGION</var>: Required. [Compute Engine
  region](https://docs.cloud.google.com/compute/docs/regions-zones#available).
- <var class="edit" scope="TEMPLATE_VERSION" translate="no">TEMPLATE_VERSION</var>: Required. Specify `latest` for the latest template version, or the date of a specific version, for example, `2023-03-17_v0.1.0-beta` (visit [gs://dataproc-templates-binaries](https://console.cloud.google.com/storage/browser/dataproc-templates-binaries) or run `gcloud storage ls gs://dataproc-templates-binaries` to list available template versions).
- <var class="edit" scope="SUBNET" translate="no">SUBNET</var>: Optional. If a subnet is not specified, the subnet in the specified REGION in the `default` network is selected.

  **Example:** `projects/PROJECT_ID/regions/REGION/subnetworks/SUBNET_NAME`
- <var class="edit" scope="JDBC_CONNECTOR_CLOUD_STORAGE_PATH" translate="no">JDBC_CONNECTOR_CLOUD_STORAGE_PATH</var>: Required. The full Cloud Storage path, including the filename, where the JDBC connector jar is stored. You can use the following commands to download JDBC connectors for uploading to Cloud Storage:
  - **MySQL:**

        wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.30.tar.gz

  - **Postgres SQL:**

        wget https://jdbc.postgresql.org/download/postgresql-42.2.6.jar

  - **Microsoft SQL Server:**

          
        wget https://repo1.maven.org/maven2/com/microsoft/sqlserver/mssql-jdbc/6.4.0.jre8/mssql-jdbc-6.4.0.jre8.jar

  - **Oracle:**

        wget https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/21.7.0.0/ojdbc8-21.7.0.0.jar

- The following variables are used to construct the required <var class="edit" scope="JDBC_CONNECTION_URL" translate="no">JDBC_CONNECTION_URL</var>:
  - <var class="edit" scope="JDBC_HOST" translate="no">JDBC_HOST</var>, <var class="edit" scope="JDBC_PORT" translate="no">JDBC_PORT</var>, <var class="edit" scope="JDBC_DATABASE" translate="no">JDBC_DATABASE</var>, or, for Oracle, <var class="edit" scope="JDBC_SERVICE" translate="no">JDBC_SERVICE</var>, <var class="edit" scope="JDBC_USERNAME" translate="no">JDBC_USERNAME</var>, and <var class="edit" scope="JDBC_PASSWORD" translate="no">JDBC_PASSWORD</var>: Required. JDBC host, port, database, username, and password.
    - **MySQL:**

          jdbc:mysql://JDBC_HOST:JDBC_PORT/JDBC_DATABASE?user=JDBC_USERNAME&password=JDBC_PASSWORD

    - **PostgreSQL:**

          jdbc:postgresql://JDBC_HOST:JDBC_PORT/JDBC_DATABASE?user=JDBC_USERNAME&password=JDBC_PASSWORD

    - **Microsoft SQL Server:**

          jdbc:sqlserver://JDBC_HOST:JDBC_PORT;databaseName=JDBC_DATABASE;user=JDBC_USERNAME;password=JDBC_PASSWORD

    - **Oracle:**

          jdbc:oracle:thin:@//JDBC_HOST:JDBC_PORT/JDBC_SERVICE?user=JDBC_USERNAME&password=JDBC_PASSWORD

  - <var class="edit" scope="DRIVER" translate="no">DRIVER</var>: Required. The JDBC driver which will be used for the connection:
    - **MySQL:**

          com.mysql.cj.jdbc.Driver

    - **Postgres SQL:**

          org.postgresql.Driver

    - **Microsoft SQL Server:**

            com.microsoft.sqlserver.jdbc.SQLServerDriver

    - **Oracle:**

          oracle.jdbc.driver.OracleDriver

  - <var class="edit" scope="QUERY" translate="no">QUERY</var> or <var class="edit" scope="QUERY_FILE" translate="no">QUERY_FILE</var>: Required. Set **either** `QUERY` or `QUERY_FILE` to specify the query to use to extract data from JDBC
  - <var class="edit" scope="INPUT_PARTITION_COLUMN" translate="no">INPUT_PARTITION_COLUMN</var>, <var class="edit" scope="LOWERBOUND" translate="no">LOWERBOUND</var>, <var class="edit" scope="UPPERBOUND" translate="no">UPPERBOUND</var>, <var class="edit" scope="NUM_PARTITIONS" translate="no">NUM_PARTITIONS</var>: Optional. If used, all of the following parameters must be specified:
    - **INPUT_PARTITION_COLUMN**: JDBC input table partition column name.
    - **LOWERBOUND**: JDBC input table partition column lower bound used to determine the partition stride.
    - **UPPERBOUND:** JDBC input table partition column upper bound used to decide the partition stride.
    - **NUM_PARTITIONS:** The maximum number of partitions that can be used for parallelism of table reads and writes. If specified, this value is used for the JDBC input and output connection. Default: `10`.
  - <var class="edit" scope="FETCHSIZE" translate="no">FETCHSIZE</var>: Optional. How many rows to fetch per round trip. Default: 10.
  - <var class="edit" scope="JDBC_SESSION_INIT" translate="no">JDBC_SESSION_INIT</var>: Optional. Session initialization statement to read Java templates.
  - <var class="edit" scope="TEMPVIEW" translate="no">TEMPVIEW</var> and <var class="edit" scope="SQL_QUERY" translate="no">SQL_QUERY</var>: Optional. You can use these two optional parameters to apply a Spark SQL transformation while loading data into Spanner. TEMPVIEW is the temporary view name, and SQL_QUERY is the query statement. TEMPVIEW and the table name in SQL_QUERY must match.
  - <var class="edit" scope="INSTANCE" translate="no">INSTANCE</var>: Required. Spanner instance ID.
  - <var class="edit" scope="SPANNER_DATABASE" translate="no">SPANNER_DATABASE</var>: Required. Spanner database ID.
  - <var class="edit" scope="TABLE" translate="no">TABLE</var>: Required. Spanner output table name.
  - <var class="edit" scope="SPANNER_JDBC_DIALECT" translate="no">SPANNER_JDBC_DIALECT</var>: Required. Spanner JDBC dialect. Options: `googlesql` or `postgresql`. Defaults to `googlesql`.
  - <var class="edit" scope="MODE" translate="no">MODE</var>: Optional. Write mode for Spanner output. Options: `Append`, `Overwrite`, `Ignore`, or `ErrorIfExists`. Defaults to `ErrorIfExists`.
  - <var class="edit" scope="PRIMARY_KEY" translate="no">PRIMARY_KEY</var>: Required. Comma separated Primary key columns needed when creating Spanner output table.
  - <var class="edit" scope="SERVICE_ACCOUNT" translate="no">SERVICE_ACCOUNT</var>: Optional. If not provided, the [default Compute Engine service account](https://docs.cloud.google.com/compute/docs/access/service-accounts.md.txt#default_service_account) is used.
  - <var class="edit" scope="PROPERTY_NAME" translate="no">PROPERTY</var> and <var class="edit" scope="PROPERTY_VALUE" translate="no">PROPERTY_VALUE</var>: Optional. Comma-separated list of [Spark property](https://docs.cloud.google.com/managed-spark/docs/concepts/properties)=`value` pairs.
  - <var class="edit" scope="LABEL" translate="no">LABEL</var> and <var class="edit" scope="LABEL_VALUE" translate="no">LABEL_VALUE</var>: Optional. Comma-separated list of `label`=`value` pairs.
  - <var class="edit" scope="LOG_LEVEL" translate="no">LOG_LEVEL</var>: Optional. Level of logging. Can be one of `ALL`, `DEBUG`, `ERROR`, `FATAL`, `INFO`, `OFF`, `TRACE`, or `WARN`. Default: `INFO`.
  - <var class="edit" scope="KMS_KEY" translate="no">KMS_KEY</var>: Optional. The Cloud Key Management Service key to use for encryption. If a key is not specified, data is [encrypted at rest](https://docs.cloud.google.com/docs/security/encryption/default-encryption.md.txt) using a Google-owned and Google-managed encryption key.

    **Example:** `projects/PROJECT_ID/regions/REGION/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME`


  Execute the

  following

  command:

  #### Linux, macOS, or Cloud Shell

  > [!NOTE]
  > **Note:** Ensure you have initialized the Google Cloud CLI with authentication and a project by running either [gcloud init](https://docs.cloud.google.com/sdk/gcloud/reference/init); or [gcloud auth login](https://docs.cloud.google.com/sdk/gcloud/reference/auth/login.md.txt) and [gcloud config set project](https://docs.cloud.google.com/sdk/gcloud/reference/config/set.md.txt).

  ```bash
  gcloud dataproc batches submit spark \
      --class=com.google.cloud.dataproc.templates.main.DataProcTemplate \
      --version="2.3" \
      --project="PROJECT_ID" \
      --region="REGION" \
      --jars="gs://dataproc-templates-binaries/TEMPLATE_VERSION/java/dataproc-templates.jar,JDBC_CONNECTOR_CLOUD_STORAGE_PATH" \
      --subnet="SUBNET" \
      --kms-key="KMS_KEY" \
      --service-account="SERVICE_ACCOUNT" \
      --properties="PROPERTY=PROPERTY_VALUE" \
      --labels="LABEL=LABEL_VALUE" \
      -- --template=JDBCTOSPANNER \
      --templateProperty log.level="LOG_LEVEL" \
      --templateProperty project.id="PROJECT_ID" \
      --templateProperty jdbctospanner.jdbc.url="JDBC_CONNECTION_URL" \
      --templateProperty jdbctospanner.jdbc.driver.class.name="DRIVER" \
      --templateProperty jdbctospanner.jdbc.fetchsize="FETCHSIZE" \
      --templateProperty jdbctospanner.jdbc.sessioninitstatement="JDBC_SESSION_INIT" \
      --templateProperty jdbctospanner.sql="QUERY" \
      --templateProperty jdbctospanner.sql.file="QUERY_FILE" \
      --templateProperty jdbctospanner.sql.numPartitions="NUM_PARTITIONS" \
      --templateProperty jdbctospanner.sql.partitionColumn="INPUT_PARTITION_COLUMN" \
      --templateProperty jdbctospanner.sql.lowerBound="LOWERBOUND" \
      --templateProperty jdbctospanner.sql.upperBound="UPPERBOUND" \
      --templateProperty jdbctospanner.output.instance="INSTANCE" \
      --templateProperty jdbctospanner.output.database="SPANNER_DATABASE" \
      --templateProperty jdbctospanner.output.table="TABLE" \
      --templateProperty jdbctospanner.output.saveMode="MODE" \
      --templateProperty jdbctospanner.output.primaryKey="PRIMARY_KEY" \
      --templateProperty jdbctospanner.output.batch.size="BATCHSIZE" \
      --templateProperty jdbctospanner.temp.table="TEMPVIEW" \
      --templateProperty jdbctospanner.temp.query="SQL_QUERY" \
      --templateProperty spanner.jdbc.dialect="SPANNER_JDBC_DIALECT" 
  ```

  #### Windows (PowerShell)

  > [!NOTE]
  > **Note:** Ensure you have initialized the Google Cloud CLI with authentication and a project by running either [gcloud init](https://docs.cloud.google.com/sdk/gcloud/reference/init); or [gcloud auth login](https://docs.cloud.google.com/sdk/gcloud/reference/auth/login.md.txt) and [gcloud config set project](https://docs.cloud.google.com/sdk/gcloud/reference/config/set.md.txt).

  ```bash
  gcloud dataproc batches submit spark `
      --class=com.google.cloud.dataproc.templates.main.DataProcTemplate `
      --version="2.3" `
      --project="PROJECT_ID" `
      --region="REGION" `
      --jars="gs://dataproc-templates-binaries/TEMPLATE_VERSION/java/dataproc-templates.jar,JDBC_CONNECTOR_CLOUD_STORAGE_PATH" `
      --subnet="SUBNET" `
      --kms-key="KMS_KEY" `
      --service-account="SERVICE_ACCOUNT" `
      --properties="PROPERTY=PROPERTY_VALUE" `
      --labels="LABEL=LABEL_VALUE" `
      -- --template=JDBCTOSPANNER `
      --templateProperty log.level="LOG_LEVEL" `
      --templateProperty project.id="PROJECT_ID" `
      --templateProperty jdbctospanner.jdbc.url="JDBC_CONNECTION_URL" `
      --templateProperty jdbctospanner.jdbc.driver.class.name="DRIVER" `
      --templateProperty jdbctospanner.jdbc.fetchsize="FETCHSIZE" `
      --templateProperty jdbctospanner.jdbc.sessioninitstatement="JDBC_SESSION_INIT" `
      --templateProperty jdbctospanner.sql="QUERY" `
      --templateProperty jdbctospanner.sql.file="QUERY_FILE" `
      --templateProperty jdbctospanner.sql.numPartitions="NUM_PARTITIONS" `
      --templateProperty jdbctospanner.sql.partitionColumn="INPUT_PARTITION_COLUMN" `
      --templateProperty jdbctospanner.sql.lowerBound="LOWERBOUND" `
      --templateProperty jdbctospanner.sql.upperBound="UPPERBOUND" `
      --templateProperty jdbctospanner.output.instance="INSTANCE" `
      --templateProperty jdbctospanner.output.database="SPANNER_DATABASE" `
      --templateProperty jdbctospanner.output.table="TABLE" `
      --templateProperty jdbctospanner.output.saveMode="MODE" `
      --templateProperty jdbctospanner.output.primaryKey="PRIMARY_KEY" `
      --templateProperty jdbctospanner.output.batch.size="BATCHSIZE" `
      --templateProperty jdbctospanner.temp.table="TEMPVIEW" `
      --templateProperty jdbctospanner.temp.query="SQL_QUERY" `
      --templateProperty spanner.jdbc.dialect="SPANNER_JDBC_DIALECT" 
  ```

  #### Windows (cmd.exe)

  > [!NOTE]
  > **Note:** Ensure you have initialized the Google Cloud CLI with authentication and a project by running either [gcloud init](https://docs.cloud.google.com/sdk/gcloud/reference/init); or [gcloud auth login](https://docs.cloud.google.com/sdk/gcloud/reference/auth/login.md.txt) and [gcloud config set project](https://docs.cloud.google.com/sdk/gcloud/reference/config/set.md.txt).

  ```bash
  gcloud dataproc batches submit spark ^
      --class=com.google.cloud.dataproc.templates.main.DataProcTemplate ^
      --version="2.3" ^
      --project="PROJECT_ID" ^
      --region="REGION" ^
      --jars="gs://dataproc-templates-binaries/TEMPLATE_VERSION/java/dataproc-templates.jar,JDBC_CONNECTOR_CLOUD_STORAGE_PATH" ^
      --subnet="SUBNET" ^
      --kms-key="KMS_KEY" ^
      --service-account="SERVICE_ACCOUNT" ^
      --properties="PROPERTY=PROPERTY_VALUE" ^
      --labels="LABEL=LABEL_VALUE" ^
      -- --template=JDBCTOSPANNER ^
      --templateProperty log.level="LOG_LEVEL" ^
      --templateProperty project.id="PROJECT_ID" ^
      --templateProperty jdbctospanner.jdbc.url="JDBC_CONNECTION_URL" ^
      --templateProperty jdbctospanner.jdbc.driver.class.name="DRIVER" ^
      --templateProperty jdbctospanner.jdbc.fetchsize="FETCHSIZE" ^
      --templateProperty jdbctospanner.jdbc.sessioninitstatement="JDBC_SESSION_INIT" ^
      --templateProperty jdbctospanner.sql="QUERY" ^
      --templateProperty jdbctospanner.sql.file="QUERY_FILE" ^
      --templateProperty jdbctospanner.sql.numPartitions="NUM_PARTITIONS" ^
      --templateProperty jdbctospanner.sql.partitionColumn="INPUT_PARTITION_COLUMN" ^
      --templateProperty jdbctospanner.sql.lowerBound="LOWERBOUND" ^
      --templateProperty jdbctospanner.sql.upperBound="UPPERBOUND" ^
      --templateProperty jdbctospanner.output.instance="INSTANCE" ^
      --templateProperty jdbctospanner.output.database="SPANNER_DATABASE" ^
      --templateProperty jdbctospanner.output.table="TABLE" ^
      --templateProperty jdbctospanner.output.saveMode="MODE" ^
      --templateProperty jdbctospanner.output.primaryKey="PRIMARY_KEY" ^
      --templateProperty jdbctospanner.output.batch.size="BATCHSIZE" ^
      --templateProperty jdbctospanner.temp.table="TEMPVIEW" ^
      --templateProperty jdbctospanner.temp.query="SQL_QUERY" ^
      --templateProperty spanner.jdbc.dialect="SPANNER_JDBC_DIALECT" 
  ```

### REST


Before using any of the request data,
make the following replacements:

- <var class="edit" scope="PROJECT_ID" translate="no">PROJECT_ID</var>: Required. Your Google Cloud project ID listed in the [IAM Settings](https://console.cloud.google.com/iam-admin/settings/project).
- <var class="edit" scope="REGION" translate="no">REGION</var>: Required. [Compute Engine
  region](https://docs.cloud.google.com/compute/docs/regions-zones#available).
- <var class="edit" scope="TEMPLATE_VERSION" translate="no">TEMPLATE_VERSION</var>: Required. Specify `latest` for the latest template version, or the date of a specific version, for example, `2023-03-17_v0.1.0-beta` (visit [gs://dataproc-templates-binaries](https://console.cloud.google.com/storage/browser/dataproc-templates-binaries) or run `gcloud storage ls gs://dataproc-templates-binaries` to list available template versions).
- <var class="edit" scope="SUBNET" translate="no">SUBNET</var>: Optional. If a subnet is not specified, the subnet in the specified REGION in the `default` network is selected.

  **Example:** `projects/PROJECT_ID/regions/REGION/subnetworks/SUBNET_NAME`
- <var class="edit" scope="JDBC_CONNECTOR_CLOUD_STORAGE_PATH" translate="no">JDBC_CONNECTOR_CLOUD_STORAGE_PATH</var>: Required. The full Cloud Storage path, including the filename, where the JDBC connector jar is stored. You can use the following commands to download JDBC connectors for uploading to Cloud Storage:
  - **MySQL:**

        wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.30.tar.gz

  - **Postgres SQL:**

        wget https://jdbc.postgresql.org/download/postgresql-42.2.6.jar

  - **Microsoft SQL Server:**

          
        wget https://repo1.maven.org/maven2/com/microsoft/sqlserver/mssql-jdbc/6.4.0.jre8/mssql-jdbc-6.4.0.jre8.jar

  - **Oracle:**

        wget https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/21.7.0.0/ojdbc8-21.7.0.0.jar

- The following variables are used to construct the required <var class="edit" scope="JDBC_CONNECTION_URL" translate="no">JDBC_CONNECTION_URL</var>:
  - <var class="edit" scope="JDBC_HOST" translate="no">JDBC_HOST</var>, <var class="edit" scope="JDBC_PORT" translate="no">JDBC_PORT</var>, <var class="edit" scope="JDBC_DATABASE" translate="no">JDBC_DATABASE</var>, or, for Oracle, <var class="edit" scope="JDBC_SERVICE" translate="no">JDBC_SERVICE</var>, <var class="edit" scope="JDBC_USERNAME" translate="no">JDBC_USERNAME</var>, and <var class="edit" scope="JDBC_PASSWORD" translate="no">JDBC_PASSWORD</var>: Required. JDBC host, port, database, username, and password.
    - **MySQL:**

          jdbc:mysql://JDBC_HOST:JDBC_PORT/JDBC_DATABASE?user=JDBC_USERNAME&password=JDBC_PASSWORD

    - **PostgreSQL:**

          jdbc:postgresql://JDBC_HOST:JDBC_PORT/JDBC_DATABASE?user=JDBC_USERNAME&password=JDBC_PASSWORD

    - **Microsoft SQL Server:**

          jdbc:sqlserver://JDBC_HOST:JDBC_PORT;databaseName=JDBC_DATABASE;user=JDBC_USERNAME;password=JDBC_PASSWORD

    - **Oracle:**

          jdbc:oracle:thin:@//JDBC_HOST:JDBC_PORT/JDBC_SERVICE?user=JDBC_USERNAME&password=JDBC_PASSWORD

  - <var class="edit" scope="DRIVER" translate="no">DRIVER</var>: Required. The JDBC driver which will be used for the connection:
    - **MySQL:**

          com.mysql.cj.jdbc.Driver

    - **Postgres SQL:**

          org.postgresql.Driver

    - **Microsoft SQL Server:**

            com.microsoft.sqlserver.jdbc.SQLServerDriver

    - **Oracle:**

          oracle.jdbc.driver.OracleDriver

  - <var class="edit" scope="QUERY" translate="no">QUERY</var> or <var class="edit" scope="QUERY_FILE" translate="no">QUERY_FILE</var>: Required. Set **either** `QUERY` or `QUERY_FILE` to specify the query to use to extract data from JDBC
  - <var class="edit" scope="INPUT_PARTITION_COLUMN" translate="no">INPUT_PARTITION_COLUMN</var>, <var class="edit" scope="LOWERBOUND" translate="no">LOWERBOUND</var>, <var class="edit" scope="UPPERBOUND" translate="no">UPPERBOUND</var>, <var class="edit" scope="NUM_PARTITIONS" translate="no">NUM_PARTITIONS</var>: Optional. If used, all of the following parameters must be specified:
    - **INPUT_PARTITION_COLUMN**: JDBC input table partition column name.
    - **LOWERBOUND**: JDBC input table partition column lower bound used to determine the partition stride.
    - **UPPERBOUND:** JDBC input table partition column upper bound used to decide the partition stride.
    - **NUM_PARTITIONS:** The maximum number of partitions that can be used for parallelism of table reads and writes. If specified, this value is used for the JDBC input and output connection. Default: `10`.
  - <var class="edit" scope="FETCHSIZE" translate="no">FETCHSIZE</var>: Optional. How many rows to fetch per round trip. Default: 10.
  - <var class="edit" scope="JDBC_SESSION_INIT" translate="no">JDBC_SESSION_INIT</var>: Optional. Session initialization statement to read Java templates.
  - <var class="edit" scope="TEMPVIEW" translate="no">TEMPVIEW</var> and <var class="edit" scope="SQL_QUERY" translate="no">SQL_QUERY</var>: Optional. You can use these two optional parameters to apply a Spark SQL transformation while loading data into Spanner. TEMPVIEW is the temporary view name, and SQL_QUERY is the query statement. TEMPVIEW and the table name in SQL_QUERY must match.
  - <var class="edit" scope="INSTANCE" translate="no">INSTANCE</var>: Required. Spanner instance ID.
  - <var class="edit" scope="SPANNER_DATABASE" translate="no">SPANNER_DATABASE</var>: Required. Spanner database ID.
  - <var class="edit" scope="TABLE" translate="no">TABLE</var>: Required. Spanner output table name.
  - <var class="edit" scope="SPANNER_JDBC_DIALECT" translate="no">SPANNER_JDBC_DIALECT</var>: Required. Spanner JDBC dialect. Options: `googlesql` or `postgresql`. Defaults to `googlesql`.
  - <var class="edit" scope="MODE" translate="no">MODE</var>: Optional. Write mode for Spanner output. Options: `Append`, `Overwrite`, `Ignore`, or `ErrorIfExists`. Defaults to `ErrorIfExists`.
  - <var class="edit" scope="PRIMARY_KEY" translate="no">PRIMARY_KEY</var>: Required. Comma separated Primary key columns needed when creating Spanner output table.
  - <var class="edit" scope="SERVICE_ACCOUNT" translate="no">SERVICE_ACCOUNT</var>: Optional. If not provided, the [default Compute Engine service account](https://docs.cloud.google.com/compute/docs/access/service-accounts.md.txt#default_service_account) is used.
  - <var class="edit" scope="PROPERTY_NAME" translate="no">PROPERTY</var> and <var class="edit" scope="PROPERTY_VALUE" translate="no">PROPERTY_VALUE</var>: Optional. Comma-separated list of [Spark property](https://docs.cloud.google.com/managed-spark/docs/concepts/properties)=`value` pairs.
  - <var class="edit" scope="LABEL" translate="no">LABEL</var> and <var class="edit" scope="LABEL_VALUE" translate="no">LABEL_VALUE</var>: Optional. Comma-separated list of `label`=`value` pairs.
  - <var class="edit" scope="LOG_LEVEL" translate="no">LOG_LEVEL</var>: Optional. Level of logging. Can be one of `ALL`, `DEBUG`, `ERROR`, `FATAL`, `INFO`, `OFF`, `TRACE`, or `WARN`. Default: `INFO`.
  - <var class="edit" scope="KMS_KEY" translate="no">KMS_KEY</var>: Optional. The Cloud Key Management Service key to use for encryption. If a key is not specified, data is [encrypted at rest](https://docs.cloud.google.com/docs/security/encryption/default-encryption.md.txt) using a Google-owned and Google-managed encryption key.

    **Example:** `projects/PROJECT_ID/regions/REGION/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME`


  HTTP method and URL:

  ```
  POST https://dataproc.googleapis.com/v1/projects/PROJECT_ID/locations/REGION/batches
  ```


  Request JSON body:

  ```
  {
    "environmentConfig": {
      "executionConfig": {
        "subnetworkUri": "SUBNET",
        "kmsKey": "KMS_KEY",
        "serviceAccount": "SERVICE_ACCOUNT"
      }
    },
    "labels": {
      "LABEL": "LABEL_VALUE"
    },
    "runtimeConfig": {
      "version": "2.3",
      "properties": {
        "PROPERTY": "PROPERTY_VALUE"
      }
    },
    "sparkBatch": {
      "mainClass": "com.google.cloud.dataproc.templates.main.DataProcTemplate",
      "args": [
        "--template","JDBCTOSPANNER",
        "--templateProperty","log.level=LOG_LEVEL",
        "--templateProperty","project.id=PROJECT_ID",
        "--templateProperty","jdbctospanner.jdbc.url=JDBC_CONNECTION_URL",
        "--templateProperty","jdbctospanner.jdbc.driver.class.name=DRIVER",
        "--templateProperty","jdbctospanner.jdbc.fetchsize=FETCHSIZE",
        "--templateProperty","jdbctospanner.jdbc.sessioninitstatement=JDBC_SESSION_INIT",
        "--templateProperty","jdbctospanner.sql=QUERY",
        "--templateProperty","jdbctospanner.sql.file=QUERY_FILE",
        "--templateProperty","jdbctospanner.sql.numPartitions=NUM_PARTITIONS",
        "--templateProperty","jdbctospanner.sql.partitionColumn=INPUT_PARTITION_COLUMN",
        "--templateProperty","jdbctospanner.sql.lowerBound=LOWERBOUND",
        "--templateProperty","jdbctospanner.sql.upperBound=UPPERBOUND",
        "--templateProperty","jdbctospanner.output.instance=INSTANCE",
        "--templateProperty","jdbctospanner.output.database=SPANNER_DATABASE",
        "--templateProperty","jdbctospanner.output.table=TABLE",
        "--templateProperty","jdbctospanner.output.saveMode=MODE",
        "--templateProperty","jdbctospanner.output.primaryKey=PRIMARY_KEY",
        "--templateProperty","jdbctospanner.output.batch.size=BATCHSIZE",
        "--templateProperty","jdbctospanner.temp.table=TEMPVIEW",
        "--templateProperty","jdbctospanner.temp.query=SQL_QUERY",
        "--templateProperty spanner.jdbc.dialect=SPANNER_JDBC_DIALECT"
      ],
      "jarFileUris": [
        "gs://dataproc-templates-binaries/TEMPLATE_VERSION/java/dataproc-templates.jar",
        "JDBC_CONNECTOR_CLOUD_STORAGE_PATH"
      ]
    }
  }
  ```

  To send your request, expand one of these options:

  #### curl (Linux, macOS, or Cloud Shell)

  > [!NOTE]
  > **Note:** The following command assumes that you have logged in to the `gcloud` CLI with your user account by running [`gcloud init`](https://docs.cloud.google.com/sdk/gcloud/reference/init) or [`gcloud auth login`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/login.md.txt) , or by using [Cloud Shell](https://docs.cloud.google.com/shell/docs.md.txt), which automatically logs you into the `gcloud` CLI . You can check the currently active account by running [`gcloud auth list`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/list.md.txt).


  Save the request body in a file named `request.json`,
  and execute the following command:

  ```
  curl -X POST \
       -H "Authorization: Bearer $(gcloud auth print-access-token)" \
       -H "Content-Type: application/json; charset=utf-8" \
       -d @request.json \
       "https://dataproc.googleapis.com/v1/projects/PROJECT_ID/locations/REGION/batches"
  ```

  #### PowerShell (Windows)

  > [!NOTE]
  > **Note:** The following command assumes that you have logged in to the `gcloud` CLI with your user account by running [`gcloud init`](https://docs.cloud.google.com/sdk/gcloud/reference/init) or [`gcloud auth login`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/login.md.txt) . You can check the currently active account by running [`gcloud auth list`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/list.md.txt).


  Save the request body in a file named `request.json`,
  and execute the following command:

  ```
  $cred = gcloud auth print-access-token
  $headers = @{ "Authorization" = "Bearer $cred" }

  Invoke-WebRequest `
      -Method POST `
      -Headers $headers `
      -ContentType: "application/json; charset=utf-8" `
      -InFile request.json `
      -Uri "https://dataproc.googleapis.com/v1/projects/PROJECT_ID/locations/REGION/batches" | Select-Object -Expand Content
  ```

  You should receive a JSON response similar to the following:

  ```
  {
    "name": "projects/PROJECT_ID/regions/REGION/operations/OPERATION_ID",
    "metadata": {
      "@type": "type.googleapis.com/google.cloud.dataproc.v1.BatchOperationMetadata",
      "batch": "projects/PROJECT_ID/locations/REGION/batches/BATCH_ID",
      "batchUuid": "de8af8d4-3599-4a7c-915c-798201ed1583",
      "createTime": "2023-02-24T03:31:03.440329Z",
      "operationType": "BATCH",
      "description": "Batch"
    }
  }
  ```

<br />