The Workflows connector defines the built-in
functions that can be used to access other Google Cloud products within a
workflow.

This page provides an overview of the individual connector.
There is no need to import or load connector libraries in a workflow---connectors
work out of the box when used in a call step.

## BigQuery API

A data platform for customers to create, manage, share and query data.
To learn more, see the [BigQuery API documentation](https://cloud.google.com/bigquery/).

You can use a page token in a workflow to
[paginate through the results of a query to a BigQuery dataset](https://docs.cloud.google.com/workflows/docs/paginate-bigquery-results).

## BigQuery connector sample

### YAML

    # This workflow demonstrates how to use the BigQuery connector:
    # Create a dataset and then insert a table with data from a public dataset
    # Delete both the table and dataset
    # Expected output: "SUCCESS"
    - init:
        assign:
          - project_id: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
          - dataset_id: "example_dataset"
          - table_id: "example_table"
          - query: "SELECT * FROM `bigquery-public-data.usa_names.usa_1910_2013` LIMIT 5000;"
          - create_disposition: "CREATE_IF_NEEDED"  # creates table if it doesn't exist
          - write_disposition: "WRITE_TRUNCATE"  # truncates table if it already exists
    - create_dataset:
        call: googleapis.bigquery.v2.datasets.insert
        args:
          projectId: ${project_id}
          body:
            datasetReference:
              datasetId: ${dataset_id}
              projectId: ${project_id}
            access[].role: "roles/bigquery.dataViewer"
            access[].specialGroup: "projectReaders"
    - insert_table_into_dataset:
        call: googleapis.bigquery.v2.jobs.insert
        args:
          projectId: ${project_id}
          body:
            configuration:
              query:
                query: ${query}
                destinationTable:
                  projectId: ${project_id}
                  datasetId: ${dataset_id}
                  tableId: ${table_id}
                create_disposition: ${create_disposition}
                write_disposition: ${write_disposition}
                allowLargeResults: true
                useLegacySql: false
    - delete_table_from_dataset:
        call: googleapis.bigquery.v2.tables.delete
        args:
          projectId: ${project_id}
          datasetId: ${dataset_id}
          tableId: ${table_id}
    - delete_dataset:
        call: googleapis.bigquery.v2.datasets.delete
        args:
          projectId: ${project_id}
          datasetId: ${dataset_id}
    - the_end:
        return: "SUCCESS"

### JSON

    [
      {
        "init": {
          "assign": [
            {
              "project_id": "${sys.get_env(\"GOOGLE_CLOUD_PROJECT_ID\")}"
            },
            {
              "dataset_id": "example_dataset"
            },
            {
              "table_id": "example_table"
            },
            {
              "query": "SELECT * FROM `bigquery-public-data.usa_names.usa_1910_2013` LIMIT 5000;"
            },
            {
              "create_disposition": "CREATE_IF_NEEDED"
            },
            {
              "write_disposition": "WRITE_TRUNCATE"
            }
          ]
        }
      },
      {
        "create_dataset": {
          "call": "googleapis.bigquery.v2.datasets.insert",
          "args": {
            "projectId": "${project_id}",
            "body": {
              "datasetReference": {
                "datasetId": "${dataset_id}",
                "projectId": "${project_id}"
              },
              "access[].role": "roles/bigquery.dataViewer",
              "access[].specialGroup": "projectReaders"
            }
          }
        }
      },
      {
        "insert_table_into_dataset": {
          "call": "googleapis.bigquery.v2.jobs.insert",
          "args": {
            "projectId": "${project_id}",
            "body": {
              "configuration": {
                "query": {
                  "query": "${query}",
                  "destinationTable": {
                    "projectId": "${project_id}",
                    "datasetId": "${dataset_id}",
                    "tableId": "${table_id}"
                  },
                  "create_disposition": "${create_disposition}",
                  "write_disposition": "${write_disposition}",
                  "allowLargeResults": true,
                  "useLegacySql": false
                }
              }
            }
          }
        }
      },
      {
        "delete_table_from_dataset": {
          "call": "googleapis.bigquery.v2.tables.delete",
          "args": {
            "projectId": "${project_id}",
            "datasetId": "${dataset_id}",
            "tableId": "${table_id}"
          }
        }
      },
      {
        "delete_dataset": {
          "call": "googleapis.bigquery.v2.datasets.delete",
          "args": {
            "projectId": "${project_id}",
            "datasetId": "${dataset_id}"
          }
        }
      },
      {
        "the_end": {
          "return": "SUCCESS"
        }
      }
    ]

## Module: googleapis.bigquery.v2.datasets

| Functions ||
|---|---|
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/datasets/delete` | Deletes the dataset specified by the datasetId value. Before you can delete a dataset, you must delete all its tables, either manually or by specifying deleteContents. Immediately after deletion, you can create another dataset with the same name. # IAM Permissions Requires the `bigquery.datasets.delete` permission on the dataset. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/datasets/get` | Returns the dataset specified by datasetID. # IAM Permissions Requires the `bigquery.datasets.get` permission on the dataset. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/datasets/insert` | Creates a new empty dataset. # IAM Permissions Requires the `bigquery.datasets.create` permission on the project. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/datasets/list` | Lists all datasets in the specified project to which the user has been granted the READER dataset role. # IAM Permissions Requires no specific IAM permission(s) to use this method. Results are filtered to only include datasets on which the caller has the `bigquery.datasets.get` permission. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/datasets/patch` | Updates information in an existing dataset. The update method replaces the entire dataset resource, whereas the patch method only replaces fields that are provided in the submitted dataset resource. This method supports RFC5789 patch semantics. # IAM Permissions Requires the following IAM permission(s) to use this method: - `bigquery.datasets.update` on the dataset. - `bigquery.datasets.get` on the dataset. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/datasets/undelete` | Undeletes a dataset which is within time travel window based on datasetId. If a time is specified, the dataset version deleted at that time is undeleted, else the last live version is undeleted. # IAM Permissions Requires the following IAM permission(s) to use this method: - `bigquery.datasets.create` on the project. - `bigquery.datasets.get` on the dataset. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/datasets/update` | Updates information in an existing dataset. The update method replaces the entire dataset resource, whereas the patch method only replaces fields that are provided in the submitted dataset resource. # IAM Permissions Requires the `bigquery.datasets.update` permission on the dataset. |

## Module: googleapis.bigquery.v2.jobs

| Functions ||
|---|---|
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/jobs/cancel` | Requests that a job be cancelled. This call will return immediately, and the client will need to poll for the job status to see if the cancel completed successfully. Cancelled jobs may still incur costs. # IAM Permissions Requires the `bigquery.jobs.update` permission on the job resource. If the user matches the creator of the job, the `bigquery.jobs.create` permission on the project is required instead. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/jobs/delete` | Requests the deletion of the metadata of a job. This call returns when the job's metadata is deleted. # IAM Permissions Requires the `bigquery.jobs.delete` permission on the job resource. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/jobs/get` | Returns information about a specific job. Job information is available for a six month period after creation. Requires that you're the person who ran the job, or have the Is Owner project role. # IAM Permissions Requires the `bigquery.jobs.get` permission on the job resource. If the user matches the creator of the job, the `bigquery.jobs.create` permission on the project is required instead. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/jobs/getQueryResults` | RPC to get the results of a query job. # IAM Permissions Requires the following IAM permission(s) to use this method: - `bigquery.jobs.get` on the job. - `bigquery.tables.getData` on the destination table. If the user matches the creator of the job, the following IAM permission(s) are required instead: - `bigquery.jobs.create` on the project. - `bigquery.tables.getData` on the destination table. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/jobs/insert` | Starts a new asynchronous job. This API has two different kinds of endpoint URIs, as this method supports a variety of use cases. \* The *Metadata* URI is used for most interactions, as it accepts the job configuration directly. \* The *Upload* URI is ONLY for the case when you're sending both a load job configuration and a data stream together. In this case, the Upload URI accepts the job configuration and the data as two distinct multipart MIME parts. # IAM Permissions Requires the `bigquery.jobs.create` permission on the project resource. Additional permissions are required depending on the job type: - **Load, Export, and Copy jobs** : Generally require data-level permissions such as `bigquery.tables.export` or access to external storage. - **Query jobs**: Permissions are dependent on the SQL statement. Complex queries (DDL, DCL) may require additional permissions to create reservations, modify IAM policies, or update project settings. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/jobs/list` | Lists all jobs that you started in the specified project. Job information is available for a six month period after creation. The job list is sorted in reverse chronological order, by job creation time. Requires the Can View project role, or the Is Owner project role if you set the allUsers property. # IAM Permissions Requires no specific IAM permission(s) to use this method. Users are able to list the jobs they created. Additional access is granted based on the following permissions: - Users with the `bigquery.jobs.listAll` permission can list all jobs with all metadata. - Users with the `bigquery.jobs.list` permission can list all jobs, but with redacted information for jobs they did not create. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/jobs/query` | Runs a BigQuery SQL query synchronously and returns query results if the query completes within a specified timeout. # IAM Permissions Requires the `bigquery.jobs.create` permission on the project resource. Data-level permissions are highly dependent on the SQL statement being executed. While standard queries require data access (such as `bigquery.tables.getData`), complex operations like DDL or DCL may require permissions to manage reservations, IAM policies, or project settings. |

## Module: googleapis.bigquery.v2.models

| Functions ||
|---|---|
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/models/delete` | Deletes the model specified by modelId from the dataset. # IAM Permissions Requires the `bigquery.models.delete` permission on the model. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/models/get` | Gets the specified model resource by model ID. # IAM Permissions Requires the `bigquery.models.getMetadata` permission on the model. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/models/list` | Lists all models in the specified dataset. Requires the READER dataset role. After retrieving the list of models, you can get information about a particular model by calling the models.get method. # IAM Permissions Requires the `bigquery.models.list` permission on the dataset. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/models/patch` | Patch specific fields in the specified model. # IAM Permissions Requires the `bigquery.models.updateMetadata` permission on the model. |

## Module: googleapis.bigquery.v2.projects

| Functions ||
|---|---|
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/projects/getServiceAccount` | RPC to get the service account for a project used for interactions with Google Cloud KMS. Requires the `bigquery.jobs.create` permission on the project resource. This permission is required to authorize the retrieval of the project's service identity for technical management tasks like encryption configuration. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/projects/list` | RPC to list projects to which the user has been granted any project role. Users of this method are encouraged to consider the [Resource Manager](https://cloud.google.com/resource-manager/docs/) API, which provides the underlying data for this method and has more capabilities. <br /> # IAM Permissions Requires no specific IAM permission(s) to use this method. The results are filtered to only include projects on which the caller has been granted a project-level role such as a BigQuery predefined IAM role or a basic role such as Viewer or Owner. |

## Module: googleapis.bigquery.v2.routines

| Functions ||
|---|---|
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/routines/delete` | Deletes the routine specified by routineId from the dataset. # IAM Permissions Requires the `bigquery.routines.delete` permission on the routine. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/routines/get` | Gets the specified routine resource by routine ID. # IAM Permissions Requires the `bigquery.routines.get` permission on the routine. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/routines/getIamPolicy` | Gets the access control policy for a resource. Returns an empty policy if the resource exists and does not have a policy set. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/routines/insert` | Creates a new routine in the dataset. # IAM Permissions Requires the `bigquery.routines.create` permission on the dataset. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/routines/list` | Lists all routines in the specified dataset. Requires the READER dataset role. # IAM Permissions Requires the `bigquery.routines.list` permission on the dataset. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/routines/setIamPolicy` | Sets the access control policy on the specified resource. Replaces any existing policy. Can return `NOT_FOUND`, `INVALID_ARGUMENT`, and `PERMISSION_DENIED` errors. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/routines/testIamPermissions` | Returns permissions that a caller has on the specified resource. If the resource does not exist, this will return an empty set of permissions, not a `NOT_FOUND` error. Note: This operation is designed to be used for building permission-aware UIs and command-line tools, not for authorization checking. This operation may "fail open" without warning. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/routines/update` | Updates information in an existing routine. The update method replaces the entire Routine resource. # IAM Permissions Requires the `bigquery.routines.update` permission on the routine. |

## Module: googleapis.bigquery.v2.rowAccessPolicies

| Functions ||
|---|---|
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/rowAccessPolicies/batchDelete` | Deletes provided row access policies. # IAM Permissions Requires the following IAM permission(s) on the table: - `bigquery.rowAccessPolicies.delete` - `bigquery.rowAccessPolicies.setIamPolicy` |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/rowAccessPolicies/delete` | Deletes a row access policy. # IAM Permissions Requires the following IAM permission(s) on the table: - `bigquery.rowAccessPolicies.delete` - `bigquery.rowAccessPolicies.setIamPolicy` |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/rowAccessPolicies/get` | Gets the specified row access policy by policy ID. # IAM Permissions Requires the `bigquery.rowAccessPolicies.get` permission on the table. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/rowAccessPolicies/getIamPolicy` | Gets the access control policy for a resource. Returns an empty policy if the resource exists and does not have a policy set. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/rowAccessPolicies/insert` | Creates a row access policy. # IAM Permissions Requires the following IAM permission(s) on the table: - `bigquery.rowAccessPolicies.create` - `bigquery.rowAccessPolicies.setIamPolicy` - `bigquery.tables.getData` |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/rowAccessPolicies/list` | Lists all row access policies on the specified table. # IAM Permissions Requires the `bigquery.rowAccessPolicies.list` permission on the table. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/rowAccessPolicies/testIamPermissions` | Returns permissions that a caller has on the specified resource. If the resource does not exist, this will return an empty set of permissions, not a `NOT_FOUND` error. Note: This operation is designed to be used for building permission-aware UIs and command-line tools, not for authorization checking. This operation may "fail open" without warning. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/rowAccessPolicies/update` | Updates a row access policy. # IAM Permissions Requires the following IAM permission(s) on the table: - `bigquery.rowAccessPolicies.update` - `bigquery.rowAccessPolicies.setIamPolicy` - `bigquery.tables.getData` |

## Module: googleapis.bigquery.v2.tabledata

| Functions ||
|---|---|
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/tabledata/insertAll` | Streams data into BigQuery one record at a time without needing to run a load job. # IAM Permissions Requires the following IAM permission(s) to use this method: - `bigquery.tables.updateData` on the table. - `bigquery.tables.get` on the table. - `bigquery.datasets.get` on the dataset. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/tabledata/list` | List the content of a table in rows. # IAM Permissions Requires the `bigquery.tables.getData` permission on the table. |

## Module: googleapis.bigquery.v2.tables

| Functions ||
|---|---|
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/tables/delete` | Deletes the table specified by tableId from the dataset. If the table contains data, all the data will be deleted. # IAM Permissions Requires the `bigquery.tables.delete` permission on the table. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/tables/get` | Gets the specified table resource by table ID. This method does not return the data in the table, it only returns the table resource, which describes the structure of this table. # IAM Permissions Requires the `bigquery.tables.get` permission on the table. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/tables/getIamPolicy` | Gets the access control policy for a resource. Returns an empty policy if the resource exists and does not have a policy set. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/tables/insert` | Creates a new, empty table in the dataset. # IAM Permissions Requires the `bigquery.tables.create` permission on the dataset. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/tables/list` | Lists all tables in the specified dataset. Requires the READER dataset role. # IAM Permissions Requires the `bigquery.tables.list` permission on the dataset. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/tables/patch` | Updates information in an existing table. The update method replaces the entire table resource, whereas the patch method only replaces fields that are provided in the submitted table resource. This method supports RFC5789 patch semantics. # IAM Permissions Requires the following IAM permission(s) on the table: - `bigquery.tables.update` - `bigquery.tables.get` |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/tables/setIamPolicy` | Sets the access control policy on the specified resource. Replaces any existing policy. Can return `NOT_FOUND`, `INVALID_ARGUMENT`, and `PERMISSION_DENIED` errors. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/tables/testIamPermissions` | Returns permissions that a caller has on the specified resource. If the resource does not exist, this will return an empty set of permissions, not a `NOT_FOUND` error. Note: This operation is designed to be used for building permission-aware UIs and command-line tools, not for authorization checking. This operation may "fail open" without warning. |
| `https://docs.cloud.google.com/workflows/docs/reference/googleapis/bigquery/v2/tables/update` | Updates information in an existing table. The update method replaces the entire Table resource, whereas the patch method only replaces fields that are provided in the submitted Table resource. # IAM Permissions Requires the `bigquery.tables.update` permission on the table. |