> [!WARNING]
>
> **Preview**
>
>
> This feature is
>
> subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the
> [Service Specific
> Terms](https://docs.cloud.google.com/terms/service-terms#1).
>
> Pre-GA features are available "as is" and might have limited support.
>
> For more information, see the
> [launch stage descriptions](https://cloud.google.com/products/#product-launch-stages).


This document explains how to use
[Compute Engine reservations](https://docs.cloud.google.com/compute/docs/instances/reservations-overview.md.txt)
to gain a high level of assurance that your batch inference jobs have the necessary virtual
machine (VM) resources to run.


Reservations are a Compute Engine feature. They help ensure that you have the resources
available to create VMs with the same hardware (memory and vCPUs) and optional resources (CPUs, GPUs,
TPUs, and Local SSD disks) whenever you need them.


When you create a reservation, Compute Engine verifies that the requested capacity is
available in the specified zone. If so, then Compute Engine reserves the resources,
creates the reservation, and the following happens:

- You can immediately consume the reserved resources, and they remain available until you delete the reservation.
- You're charged for the reserved resources at the same on-demand rate as running VMs, including any applicable discounts, until the reservation is deleted. A VM consuming a reservation doesn't incur separate charges. You're charged only for the resources outside of the reservation, such as disks or IP addresses. To learn more, see [pricing for
  reservations](https://docs.cloud.google.com/compute/docs/instances/reservations-overview#reservation-billing).

## Limitations and requirements


When using Compute Engine reservations with Agent Platform, consider the
following limitations and requirements:

- Agent Platform can only use reservations for [CPUs](https://docs.cloud.google.com/compute/docs/cpu-platforms.md.txt), [GPU VMs](https://docs.cloud.google.com/compute/docs/gpus.md.txt), or [TPUs](https://docs.cloud.google.com/tpu/docs/intro-to-tpu.md.txt) ([Preview](https://cloud.google.com/products#product-launch-stages)).
- Agent Platform can't consume reservations of VMs that have [Local SSD disks manually attached](https://docs.cloud.google.com/compute/docs/disks/local-ssd.md.txt#lssd_disk_options).
- Using Compute Engine reservations with Agent Platform is only supported for Gemini Enterprise Agent Platform serverless training, inference, and [Gemini Enterprise Agent Platform Workbench](https://docs.cloud.google.com/gemini-enterprise-agent-platform/notebooks/workbench/instances/reservations.md.txt) (Preview).
- A reservation's VM properties must match exactly with your Agent Platform workload to consume the reservation. For example, if a reservation specifies an `a2-ultragpu-8g` machine type, then the Agent Platform workload can only consume the reservation if it also uses an `a2-ultragpu-8g` machine type. See [Requirements](https://docs.cloud.google.com/compute/docs/instances/reservations-overview.md.txt#requirements).
- To consume a shared reservation of GPU VMs or TPUs, you must consume it using its owner project or a consumer project with which the reservation is shared. See [How shared reservations work](https://docs.cloud.google.com/compute/docs/instances/reservations-overview.md.txt#how-shared-reservations-work).
- To consume a `SPECIFIC_RESERVATION` reservation, grant the [Compute Viewer](https://docs.cloud.google.com/iam/docs/roles-permissions/compute.md.txt#compute.viewer) IAM role to the Agent Platform service account in the project that owns the reservations (`service-${PROJECT_NUMBER}@gcp-sa-aiplatform.iam.gserviceaccount.com`, where <var translate="no">PROJECT_NUMBER</var> is the project number of the project that consumes the reservation).
- The following services and capabilities aren't supported when using Compute Engine
  reservations with Agent Platform batch inference:

  - Federal Risk and Authorization Management Program (FedRAMP) compliance

## Billing

When using Compute Engine reservations, you're billed for the following:

- Compute Engine pricing for the Compute Engine resources, including any applicable committed use discounts (CUDs). See [Compute Engine pricing](https://cloud.google.com/compute/all-pricing).
- Agent Platform batch inference management fees in addition to your infrastructure usage. See [Prediction pricing](https://cloud.google.com/products/gemini-enterprise-agent-platform/pricing#prediction-prices).

## Before you begin

- Review the [requirements](https://docs.cloud.google.com/compute/docs/instances/reservations-overview.md.txt#requirements) and [restrictions](https://docs.cloud.google.com/compute/docs/instances/reservations-overview.md.txt#restrictions) for reservations.
- Review the [quota
  requirements](https://docs.cloud.google.com/compute/docs/instances/reservations-overview#requirements-shared-reservations) and [restrictions](https://docs.cloud.google.com/compute/docs/instances/reservations-overview.md.txt#restrictions-shared-reservations) for shared reservations.

## Allow a reservation to be consumed


Before consuming a reservation of CPUs, GPU VMs, or TPUs, you must set its
[sharing policy](https://docs.cloud.google.com/compute/docs/instances/reservations-overview.md.txt#sharing-policy) to allow
Agent Platform to consume the reservation. To do so, use one of the following methods:

- [Allow consumption while creating a reservation](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/use-reservations-batch.md.txt#allow-new-reservation)
- [Allow consumption in an existing reservation](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/use-reservations-batch.md.txt#allow-existing-reservation)

<br />

### Allow consumption while creating a reservation


When creating a
[single-project](https://docs.cloud.google.com/compute/docs/instances/reservations-single-project.md.txt) or
[shared reservation](https://docs.cloud.google.com/compute/docs/instances/reservations-shared.md.txt) of GPU VMs, you can
allow Agent Platform to consume the reservation as follows:

- If you're using the Google Cloud console, then, in the **Google Cloud services** section, select **Share reservation**.
- If you're using the Google Cloud CLI, then include the `--reservation-sharing-policy` flag set to `ALLOW_ALL`.
- If you're using the REST API, then, in the request body, include the `serviceShareType` field set to `ALLOW_ALL`.

<br />

### Allow consumption of an existing reservation

You can modify an auto-created reservation of GPU VMs or TPUs for a future reservation only after
the reservation's start time.

To allow Agent Platform to consume an existing reservation, use one of the following
methods:

- [Modify a
  reservation of GPU VMs](https://docs.cloud.google.com/compute/docs/instances/reservations-modify#modify-sharing-policy)
- [Modify a reservation of TPUs](https://docs.cloud.google.com/tpu/docs/share-reservation.md.txt#share-with-vertex)

<br />

## Verify that a reservation is consumed

To verify that the reservation is being consumed, see [Verify reservations consumption](https://docs.cloud.google.com/compute/docs/instances/reservations-consume.md.txt#verify-consumption) in the Compute Engine documentation.

## Get batch inferences by using a reservation

To create a batch inference request that consumes
a Compute Engine reservation of GPU VMs, you can use the REST API and
choose either Cloud Storage or BigQuery for the
source and destination.

### Cloud Storage


Before using any of the request data,
make the following replacements:

- <var translate="no">LOCATION_ID</var>: The region where the model is stored and the batch prediction job
  is executed. For example, `us-central1`.

- <var translate="no">PROJECT_ID</var>: The project where the reservation was created. To consume a shared
  reservation from another project, you must share the reservation with that project. For more
  information, see
  [Modify the consumer projects in a shared reservation](https://docs.cloud.google.com/compute/docs/instances/reservations-modify.md.txt#modifying-consumer-projects-shared-reservation).

- <var translate="no">BATCH_JOB_NAME</var>: A display name for the batch prediction job.

- <var translate="no">MODEL_ID</var>: The ID for the model to use for making predictions.

- <var translate="no">INPUT_FORMAT</var>: The [format of your input
  data](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/use-reservations-batch#input_data_requirements): `jsonl`, `csv`, `tf-record`,
  `tf-record-gzip`, or `file-list`.

- <var translate="no">INPUT_URI</var>: The Cloud Storage URI of your input data. May contain wildcards.

- <var translate="no">OUTPUT_DIRECTORY</var>: The Cloud Storage URI of a directory where you
  want Agent Platform to save output.

- <var translate="no">MACHINE_TYPE</var>: The [machine resources](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/configure-compute.md.txt)
  to be used for this batch prediction job.

- <var translate="no">ACCELERATOR_TYPE</var>: The type of accelerator to attach to the machine. For more
  information about the type of GPU that each machine type supports, see
  [GPUs for compute workloads](https://docs.cloud.google.com/compute/docs/gpus.md.txt#gpus_for_compute_workloads).

- <var translate="no">ACCELERATOR_COUNT</var>: The number of accelerators to attach to the machine.

-
  <var translate="no">RESERVATION_AFFINITY_TYPE</var>: Must be `ANY`,
  `SPECIFIC_RESERVATION`, or `NONE`.

  - `ANY` means that the VMs of your `customJob` can automatically consume any reservation with matching properties.
  - `SPECIFIC_RESERVATION` means that the VMs of your `customJob` can only consume a reservation that the VMs specifically targets by name.
  - `NONE` means that the VMs of your `customJob` can't consume any reservation. Specifying `NONE` has the same effect as omitting a reservation affinity specification.
- <var translate="no">BATCH_SIZE</var>: The number of instances to send in each prediction
  request; the default is 64. Increasing the batch size can lead to higher
  throughput, but it can also cause request timeouts.

- <var translate="no">STARTING_REPLICA_COUNT</var>: The number of nodes for this batch prediction job.


HTTP method and URL:

```
POST https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs
```


Request JSON body:

```
{
  "displayName": "BATCH_JOB_NAME",
  "model": "projects/PROJECT_ID/locations/LOCATION_ID/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "INPUT_FORMAT",
    "gcsSource": {
      "uris": ["INPUT_URI"],
    },
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "OUTPUT_DIRECTORY",
    },
  },
  "dedicatedResources" : {
    "machineSpec" : {
      "machineType": MACHINE_TYPE,
      "acceleratorType": "ACCELERATOR_TYPE",
      "acceleratorCount": ACCELERATOR_COUNT,
      "reservationAffinity": {
        "reservationAffinityType": "RESERVATION_AFFINITY_TYPE",
        "key": "compute.googleapis.com/reservation-name",
        "values": [
          "projects/PROJECT_ID/zones/ZONE/reservations/RESERVATION_NAME"
        ]
      }
    },
    "startingReplicaCount": STARTING_REPLICA_COUNT
  },
  "manualBatchTuningParameters": {
    "batch_size": BATCH_SIZE,
  }
}
```

To send your request, choose one of these options:

#### curl

> [!NOTE]
> **Note:** The following command assumes that you have logged in to the `gcloud` CLI with your user account by running [`gcloud init`](https://docs.cloud.google.com/sdk/gcloud/reference/init) or [`gcloud auth login`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/login.md.txt) , or by using [Cloud Shell](https://docs.cloud.google.com/shell/docs.md.txt), which automatically logs you into the `gcloud` CLI . You can check the currently active account by running [`gcloud auth list`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/list.md.txt).


Save the request body in a file named `request.json`,
and execute the following command:

```
curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs"
```

#### PowerShell

> [!NOTE]
> **Note:** The following command assumes that you have logged in to the `gcloud` CLI with your user account by running [`gcloud init`](https://docs.cloud.google.com/sdk/gcloud/reference/init) or [`gcloud auth login`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/login.md.txt) . You can check the currently active account by running [`gcloud auth list`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/list.md.txt).


Save the request body in a file named `request.json`,
and execute the following command:

```
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs" | Select-Object -Expand Content
```

You should receive a JSON response similar to the following:

```
{
  "name": "projects/PROJECT_NUMBER/locations/LOCATION_ID/batchPredictionJobs/BATCH_JOB_ID",
  "displayName": "BATCH_JOB_NAME 202005291958",
  "model": "projects/PROJECT_ID/locations/LOCATION_ID/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "jsonl",
    "gcsSource": {
      "uris": [
        "INPUT_URI"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "OUTPUT_DIRECTORY"
    }
  },
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-05-30T02:58:44.341643Z",
  "updateTime": "2020-05-30T02:58:44.341643Z",
}
```

<br />

### BigQuery


Before using any of the request data,
make the following replacements:

- <var translate="no">LOCATION_ID</var>: The region where the model is stored and the batch prediction job
  is executed. For example, `us-central1`.

- <var translate="no">PROJECT_ID</var>: The project where the reservation was created. To consume a shared
  reservation from another project, you must share the reservation with that project. For more
  information, see
  [Modify the consumer projects in a shared reservation](https://docs.cloud.google.com/compute/docs/instances/reservations-modify.md.txt#modifying-consumer-projects-shared-reservation).

- <var translate="no">BATCH_JOB_NAME</var>: A display name for the batch prediction job.

- <var translate="no">MODEL_ID</var>: The ID for the model to use for making predictions.

- <var translate="no">INPUT_PROJECT_ID</var>: The ID of the Google Cloud project where you
  want to get the data from.

- <var translate="no">INPUT_DATASET_NAME</var>: The name of the BigQuery dataset
  where you want to get the data from.

- <var translate="no">INPUT_TABLE_NAME</var>: The name of the BigQuery table where
  you want to get the data from.

- <var translate="no">OUTPUT_PROJECT_ID</var>: The ID of the Google Cloud project where
  you want to save the output.

- <var translate="no">OUTPUT_DATASET_NAME</var>: The name of the destination
  BigQuery dataset where you want to save the output.

- <var translate="no">OUTPUT_TABLE_NAME</var>: The name of the BigQuery destination
  table where you want to save the output.

- <var translate="no">MACHINE_TYPE</var>: The [machine resources](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/configure-compute.md.txt)
  to be used for this batch prediction job.

- <var translate="no">ACCELERATOR_TYPE</var>: The type of accelerator to attach to the machine. For more
  information about the type of GPU that each machine type supports, see
  [GPUs for compute workloads](https://docs.cloud.google.com/compute/docs/gpus.md.txt#gpus_for_compute_workloads).

- <var translate="no">ACCELERATOR_COUNT</var>: The number of accelerators to attach to the machine.

-
  <var translate="no">RESERVATION_AFFINITY_TYPE</var>: Must be `ANY`,
  `SPECIFIC_RESERVATION`, or `NONE`.

  - `ANY` means that the VMs of your `customJob` can automatically consume any reservation with matching properties.
  - `SPECIFIC_RESERVATION` means that the VMs of your `customJob` can only consume a reservation that the VMs specifically targets by name.
  - `NONE` means that the VMs of your `customJob` can't consume any reservation. Specifying `NONE` has the same effect as omitting a reservation affinity specification.
- <var translate="no">BATCH_SIZE</var>: The number of instances to send in each prediction
  request; the default is 64. Increasing the batch size can lead to higher
  throughput, but it can also cause request timeouts.

- <var translate="no">STARTING_REPLICA_COUNT</var>: The number of nodes for this batch prediction job.


HTTP method and URL:

```
POST https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs
```


Request JSON body:

```
{
  "displayName": "BATCH_JOB_NAME",
  "model": "projects/PROJECT_ID/locations/LOCATION_ID/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "bigquery",
    "bigquerySource": {
      "inputUri": "bq://INPUT_PROJECT_ID.INPUT_DATASET_NAME.INPUT_TABLE_NAME"
    },
  },
  "outputConfig": {
    "predictionsFormat":"bigquery",
    "bigqueryDestination":{
      "outputUri": "bq://OUTPUT_PROJECT_ID.OUTPUT_DATASET_NAME.OUTPUT_TABLE_NAME"
    }
  },
  "dedicatedResources" : {
    "machineSpec" : {
      "machineType": MACHINE_TYPE,
      "acceleratorType": "ACCELERATOR_TYPE",
      "acceleratorCount": ACCELERATOR_COUNT,
      "reservationAffinity": {
        "reservationAffinityType": "RESERVATION_AFFINITY_TYPE",
        "key": "compute.googleapis.com/reservation-name",
        "values": [
          "projects/PROJECT_ID/zones/ZONE/reservations/RESERVATION_NAME"
        ]
      }
    },
    "startingReplicaCount": STARTING_REPLICA_COUNT
  },
  "manualBatchTuningParameters": {
    "batch_size": BATCH_SIZE,
  }
}
```

To send your request, choose one of these options:

#### curl

> [!NOTE]
> **Note:** The following command assumes that you have logged in to the `gcloud` CLI with your user account by running [`gcloud init`](https://docs.cloud.google.com/sdk/gcloud/reference/init) or [`gcloud auth login`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/login.md.txt) , or by using [Cloud Shell](https://docs.cloud.google.com/shell/docs.md.txt), which automatically logs you into the `gcloud` CLI . You can check the currently active account by running [`gcloud auth list`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/list.md.txt).


Save the request body in a file named `request.json`,
and execute the following command:

```
curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs"
```

#### PowerShell

> [!NOTE]
> **Note:** The following command assumes that you have logged in to the `gcloud` CLI with your user account by running [`gcloud init`](https://docs.cloud.google.com/sdk/gcloud/reference/init) or [`gcloud auth login`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/login.md.txt) . You can check the currently active account by running [`gcloud auth list`](https://docs.cloud.google.com/sdk/gcloud/reference/auth/list.md.txt).


Save the request body in a file named `request.json`,
and execute the following command:

```
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs" | Select-Object -Expand Content
```

You should receive a JSON response similar to the following:

```
{
  "name": "projects/PROJECT_NUMBER/locations/LOCATION_ID/batchPredictionJobs/BATCH_JOB_ID",
  "displayName": "BATCH_JOB_NAME 202005291958",
  "model": "projects/PROJECT_ID/locations/LOCATION_ID/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "jsonl",
    "bigquerySource": {
      "uris": [
        "INPUT_URI"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "bigqueryDestination": {
      "outputUri": "OUTPUT_URI"
    }
  },
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-05-30T02:58:44.341643Z",
  "updateTime": "2020-05-30T02:58:44.341643Z",
}
```

<br />

### Retrieve batch inference results

When a batch inference task is complete, the output of the inference is stored
in the Cloud Storage bucket or BigQuery location that you
specified in your request.

## What's next

- Learn more about [reservations of Compute Engine
  zonal resources](https://docs.cloud.google.com/compute/docs/instances/reservations-overview).
- Learn how to [use reservations with Agent Platform
  online inference](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/use-reservations).
- Learn how to [use reservations with Agent Platform
  training](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/training/use-reservations).
- Learn how to [view reservations](https://docs.cloud.google.com/compute/docs/instances/reservations-view.md.txt).
- Learn how to [monitor reservations
  consumption](https://docs.cloud.google.com/compute/docs/instances/reservations-monitor).