Improve obtainability by using instance flexibility

This document describes how to create and run a Batch job that uses instance flexibility. Instance flexibility lets you allow a job to run on multiple machine types that you specify and can optionally rank.

By allowing more machine types for a job, you can use instance flexibility to improve obtainability, which is the probability that resources are available to run your job. For more information about choosing which obtainability features to use for your workload, see Improve resource obtainability. For example, using instance flexibility can help you do the following:

Reduce the probability of resource availability errors and related scheduling delays.
Select resources with higher obtainability to help jobs finish sooner—for example:
- Increase the probability of obtaining more instances for jobs that support parallel processing.
- Obtain Spot VMs that are less likely to be preempted.

Before you begin

If you haven't used Batch before, review Get started with Batch and enable Batch by completing the prerequisites for projects and users.
To get the permissions that you need to create a job, ask your administrator to grant you the following IAM roles:
- Batch Job Editor (roles/batch.jobsEditor) on the project
- Service Account User (roles/iam.serviceAccountUser) on the job's service account, which by default is the default Compute Engine service account
For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Restrictions

When a job uses instance flexibility, all of the following restrictions apply:

Obtainability improvement increases with the number of machine types that you allow a job to use.

Important: If a job doesn't have any machine type requirements, then, instead of using instance flexibility, you can improve obtainability even more by omitting the machine type entirely. However, you can't omit the machine type when creating a job through the Google Cloud console or when creating a job that uses an instance template.
The job can't specify the FLEX_START provisioning model or RESERVATION_BOUND provisioning model.
The job can't consume a specifically targeted reservation. (The reservation field must be undefined or set to NO_RESERVATION.)

A specifically targeted reservation includes all reservations for the RESERVATION_BOUND provisioning model and some reservations for the STANDARD provisioning model. These features are inherently incompatible because instance creation can only succeed for the reservation's machine type.
Each job can have up to 50 instance selections.
Each instance selection in a job must have a unique name, a unique rank, and one or more unique machine types.
Machine types can't repeat; include each machine type in only one instance selection per job.
Each job can have up to 200 unique machine types across its instance selections.
Although specifying multiple machine types in the same instance selection is optional, it is required if you want resources with higher obtainability.
- Batch attempts to select one or more machine types with the highest obtainability only when the instance selection for a rank has multiple machine types. For example, this selection is important if you want your job to have Spot VMs that are less likely to be preempted or more instances for a job that can run tasks in parallel.
- Otherwise, when the instance selection for a rank only has one machine type, instance flexibility only helps prevent resource availability errors.
The job can't include the machineType field, and the machine type for any instance template that the job specifies is ignored.
You must verify that every property that you specify for the job is compatible with every allowed machine type. Otherwise, the job might fail after being submitted because it can't create instances. For example, we recommend you verify that at least the following properties for a job are compatible with all machine types:
- location (if specified, allowedLocations[] field; otherwise location)
- compute resources per task (computeResource field)
- disks and images (volumes[] field, disk[] field, and bootDisk field)
- network (network field)
- all other instance properties (instances[] subfields):
  - minimum CPU platform (minCpuPlatform field)
  - provisioning model (provisioningModel field)
  - GPUs (accelerators[] field)
You can't override disks and minimum CPU platforms for Batch jobs. All machine types use either your specified values (in the policy subfields or an instance template) or, if unspecified, use the default values for a machine type in the highest-priority instance selection. Consequently, we recommend you do all of the following:
- Verify that the boot disk type is compatible for every allowed machine type. In summary, 4th generation and later machine types don't support persistent disks, 2nd generation and earlier machine types don't support Hyperdisk, and 3rd generation and earlier machine types use persistent boot disks by default.
  
  Key Point: You can't combine 4th generation and later machine types with 2nd generation and earlier machine types due to boot disk incompatibilities. To combine 3rd generation machine types with later machine types, ensure the boot disk type is a Hyperdisk.
- Don't specify a minimum CPU platform or optional disks unless you've verified that they are supported for every allowed machine type.

Create a job that is allowed to use multiple machine types

To create a job that is allowed to use multiple machine types, select one of the following methods.

gcloud

To configure multiple machine types while using the Google Cloud CLI to create a job, complete the following steps:

Include the instanceFlexibilityPolicy field and subfields in the allocationPolicy field in the JSON file that specifies your job's configuration details. The instanceFlexibilityPolicy field supports one or more instance selections, which each have a unique name, a unique rank, and a list of one or more machine types.
```
"allocationPolicy":{
  "instanceFlexibilityPolicy":{
    "instanceSelections":{
      "INSTANCE_SELECTION_NAME":{
        "rank": RANK,
        "machineTypes": [MACHINE_TYPES]
      }
    }
  }
}
```
Replace the following:
- INSTANCE_SELECTION_NAME: a name for the instance selection. If the job has multiple instance selections, then ensure each name in this job is unique.
- RANK: Optional: an integer of zero or greater that represents the priority for this instance selection, where the lowest value is the highest priority. If not specified, the default value is 0 (highest priority). If the job has multiple instance selections, then ensure each rank in this job is unique.
- MACHINE_TYPES: a comma-separated list of one or more machine types, which are written as strings, for this instance selection. You can include each machine type in only one instance selection per job. For example, a list of three machine types can be written as follows:
```
"MACHINE_TYPE_1", "MACHINE_TYPE_2", "MACHINE_TYPE_3"
```
  Replace MACHINE_TYPE_1, MACHINE_TYPE_2, MACHINE_TYPE_3 each with a machine type—for example, e2-micro, e2-small, and n1-standard-2.
For more information, also see the example job.
While this feature is in Preview, you must use the gcloud alpha batch jobs submit command to create and submit the job.
```
gcloud alpha batch jobs submit JOB_NAME \
  --location LOCATION \
  --config JSON_CONFIGURATION_FILE
```
Replace the following:
- JOB_NAME: the name of the job.
- LOCATION: the location of the job.
- JSON_CONFIGURATION_FILE: the path for a JSON file with the job's configuration details.

REST

To use the Batch API to create a job that uses multiple machine types, use the v1alpha jobs.create method while this feature is in Preview.

To specify multiple machine types for a job, include the instanceFlexibilityPolicy field and subfields in the allocationPolicy field in the main body in the JSON file. The instanceFlexibilityPolicy field supports one or more instance selections, which each have a unique name, a unique rank, and a list of one or more machine types.

"allocationPolicy":{
  "instanceFlexibilityPolicy":{
    "instanceSelections":{
      "INSTANCE_SELECTION_NAME":{
        "rank": RANK,
        "machineTypes": [MACHINE_TYPES]
      }
    }
  }
}

Replace the following:

INSTANCE_SELECTION_NAME: a name for the instance selection. If the job has multiple instance selections, then ensure each name in this job is unique.
RANK: Optional: an integer of zero or greater that represents the priority for this instance selection, where the lowest value is the highest priority. If not specified, the default value is 0 (highest priority). If the job has multiple instance selections, then ensure each rank in this job is unique.
MACHINE_TYPES: a comma-separated list of one or more machine types, which are written as strings, for this instance selection. You can include each machine type in only one instance selection per job. For example, a list of three machine types can be written as follows:
```
"MACHINE_TYPE_1", "MACHINE_TYPE_2", "MACHINE_TYPE_3"
```
Replace MACHINE_TYPE_1, MACHINE_TYPE_2, MACHINE_TYPE_3 each with a machine type—for example, e2-micro, e2-small, and n1-standard-2.

For more information, also see the example job.

Example job JSON configuration file

For example, when using the gcloud CLI or Batch API, you can use the following JSON configuration file to create a basic script job that has two instance selections:

The instance selection named first-choice has the top priority ("rank": 0) and three machine types (e2-micro, e2-small, and n1-standard-2).
The instance selection named second-choice has the second priority ("rank": 1) and one machine type (e2-medium).

{
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "script": {
              "text": "echo Hello World! This is task $BATCH_TASK_INDEX."
            }
          }
        ]
      },
      "taskCount": 3
    }
  ],
  "allocationPolicy":{
    "instanceFlexibilityPolicy":{
      "instanceSelections":{
        "first-choice":{
          "rank": 0,
          "machineTypes": ["e2-micro", "e2-small", "n1-standard-2"]
        },
        "second-choice":{
          "rank": 1,
          "machineTypes": ["e2-medium"]
        }
      }
    }
  }
}

What's next

If you have issues creating or running a job, see Troubleshooting.
View jobs and tasks.
Learn about more job creation options.