Create a MIG that adds GPU VMs all at once

This document explains how to create a managed instance group (MIG) to add GPU virtual machine (VM) instances all at once.

The MIG uses the bulk mode of its target size policy to add the VMs all at once. It also uses the flex-start provisioning model to improve obtainability of resources. For more information about the flex-start provisioning model, see About provisioning models.

Another method to add GPU VMs all at once in a MIG is by creating resize requests. To determine whether bulk mode or resize request is more suitable for your workload, see the Comparison of bulk mode and resize request.

If your workload doesn't require the MIG to creates VMs all at once but you still want to improve resource obtainability, use the flex-start provisioning model as described in Create a MIG with Flex-start VMs.

Before you begin

To make sure that you have sufficient GPU quota for the resources you're requesting, check your GPU quota.
To understand quota consumption, read GPU VMs and preemptible allocation quotas.
If you haven't already, set up authentication. Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:
Select the tab for how you plan to use the samples on this page:
Console

When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.
gcloud
1. Install the Google Cloud CLI. After installation, initialize the Google Cloud CLI by running the following command:
  gcloud init
  If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.
  
  Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.
2. Set a default region and zone.
REST

To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.
For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

Limitations

The following limitations apply:

Create a MIG and add GPU VMs all at once

To create the GPU VMs all at once, do the following:

Create an instance template, which is required to create a MIG. The MIG then uses this template to create each VM in the group. In the template, specify the configurations for the flex-start provisioning model.

For more information about instance templates, see About instance templates.
Create a MIG for bulk provisioning of VMs. During the MIG creation, enable the bulk provisioning capability in the MIG by setting the target size policy mode to BULK. In BULK mode, the MIG attempts to create all VMs at once to meet the target size that you set.

Create an instance template

Create an instance template that specifies a GPU machine series. Then, use the template to create a MIG.

Note: If you want to run data science or machine learning workloads, consider using a Deep Learning VM image when you create an instance template. Deep Learning VM Images is a set of prepackaged VM images that comes with machine learning frameworks and essential tools. For more information about these images, see Choose an image in the Deep Learning VM Images documentation.

Permissions required for this task

To perform this task, you must have the following permissions:

All permissions required to call the instanceTemplates.insert method.

Console

Go to the Instance templates page.

Go to Instance templates
Click Create instance template. The Create an instance template page opens.
In the Name field, enter a name for the instance template.
In the Machine configuration section, do the following:
1. Click the GPUs tab.
2. In the GPU type list, select the GPU type.
3. In the Number of GPUs list, select the number of GPUs.
4. In the Machine type section, select a machine type.
In the Provisioning model section, do the following:
1. In the VM provisioning model list, select Flex-start.
  
  Note: When you select the flex-start provisioning model, you can't use reservations. The Google Cloud console automatically selects the Don't use a reservation option in the Advanced options > Management > Reservations section.
2. To set a run duration for the VMs created through the instance template, in the Enter number of hours field, enter the number of hours. The value must be between one hour (1) and seven days (168).
Optional: To change the default value boot disk type or image, in the Boot disk section, click Change. Then, follow the prompts to change the boot disk.
Click Create.

gcloud

Create an instance template by using the instance-templates create command:

gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --image-project=IMAGE_PROJECT \
    --image-family=IMAGE_FAMILY \
    --instance-termination-action=DELETE \
    --instance-template-region=REGION \
    --machine-type=MACHINE_TYPE \
    --maintenance-policy=TERMINATE \
    --max-run-duration=RUN_DURATION \
    --provisioning-model=FLEX_START \
    --reservation-affinity=none

Replace the following:

INSTANCE_TEMPLATE_NAME: the name of the instance template.
IMAGE_PROJECT: the image project that contains the image—for example, debian-cloud. For more information about the supported image projects, see Public images.
IMAGE_FAMILY: an image family. This specifies the most recent, non-deprecated OS image. For example, if you specify debian-12, the latest version in the Debian 12 image family is used. For more information about using image families, see Image families best practices.

Note: If you want to use a specific version of the OS image, such as debian-12-bookworm-v20240701, then replace the --image-family flag with the --image flag.
REGION: the region in which to create the instance template.
MACHINE_TYPE: a GPU machine type. If you specify an N1 machine type, then include the --accelerator flag to specify the number and type of GPUs to attach to your VMs.
RUN_DURATION: the duration you want the requested VMs to run. You must format the value as the number of days, hours, minutes, or seconds followed by d, h, m, or s respectively. For example, specify 30m for 30 minutes or 1d2h3m4s for one day, two hours, three minutes, and four seconds. The value must be between 10 minutes and seven days.

REST

Create an instance template by making a POST request to the instanceTemplates.insert method:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instanceTemplates

{
  "name": "INSTANCE_TEMPLATE_NAME",
  "properties": {
    "disks": [
      {
        "boot": true,
        "initializeParams": {
          "sourceImage": "projects/IMAGE_PROJECT/global/images/IMAGE
        }
      }
    ],
    "machineType": "MACHINE_TYPE",
    "networkInterfaces": [
      {
        "network": "global/networks/default"
      }
    ],
    "reservationAffinity": {
      "consumeReservationType": "NO_RESERVATION"
    },
    "scheduling": {
      "instanceTerminationAction": "DELETE",
      "maxRunDuration": {
        "seconds": RUN_DURATION
      },
      "onHostMaintenance": "TERMINATE",
      "provisioningModel": "FLEX_START"
    }
  }
}

Replace the following:

PROJECT_ID: the ID of the project in which you want to create the instance template.
REGION: the region in which to create the instance template.
INSTANCE_TEMPLATE_NAME: the name of the instance template.
IMAGE_PROJECT: the image project that contains the image—for example, debian-cloud. For more information about the supported image projects, see Public images.
IMAGE: specify one of the following:
- A specific version of the OS image—for example, debian-12-bookworm-v20240617.
- An image family, which must be formatted as family/IMAGE_FAMILY. This specifies the most recent, non-deprecated OS image. For example, if you specify family/debian-12, the latest version in the Debian 12 image family is used. For more information about using image families, see Image families best practices.
MACHINE_TYPE: a GPU machine type. If you specify an N1 machine type, then include the guestAccelerators field to specify the number and type of GPUs to attach to your VMs.
RUN_DURATION: the duration, in seconds, you want the requested VMs to run before the MIG automatically deletes them. The value must be between 600, which is 600 seconds (10 minutes), and 604800, which is 604,800 seconds (seven days).

After you create the instance template, you can view it to see its ID and review its instance properties.

Create a MIG and add GPU VMs all at once

Create a MIG as described in this section. When enabling bulk provisioning in the MIG, you must turn off repairs.

Permissions required for this task

To perform this task, you must have the following permissions:

All permissions required to call the instanceGroupManagers.insert method.

Console

Go to the Instance groups page.

Go to Instance groups
Click Create instance group. The Create instance group page opens.
In the Name field, enter a name for the MIG.
Before you select an instance template, you must turn off repairs and set the location as follows:
1. To turn off repairs, do the following:
  1. In the VM instance lifecycle section, clear the Allow repair of VMs in an alternate zone checkbox.
  2. In the Default action on failure list, select No action.
2. To set the Location section, do the following:
  - To create a zonal MIG, select Single zone.
  - To create a regional MIG, select Multiple zones. In the Target distribution shape, select Any single zone. In the dialog that appears, select Disable instance redistribution.
Go back to the Instance template field. In the Instance template list, select the instance template that you created in the previous section.
In the Number of instances field, specify the number of instances that you want to create all at once.
Select the Obtain VMs all at once with bulk target size policy checkbox.
Click Create.

gcloud

To create a MIG with bulk provisioning capability, use the instance-groups managed create command.

To create a zonal MIG, run the following command:

gcloud compute instance-groups managed create INSTANCE_GROUP_NAME \
    --default-action-on-vm-failure=do-nothing \
    --size=SIZE \
    --target-size-policy-mode=bulk \
    --template=INSTANCE_TEMPLATE_URL \
    --zone=ZONE \

To create a regional MIG, run the following command:

gcloud compute instance-groups managed create INSTANCE_GROUP_NAME \
    --default-action-on-vm-failure=do-nothing \
    --size=SIZE \
    --target-size-policy-mode=bulk \
    --template=INSTANCE_TEMPLATE_URL \
    --zones=LIST_OF_ZONES \

Replace the following:

INSTANCE_GROUP_NAME: the name of the MIG.
SIZE: the number of VMs in the MIG.
INSTANCE_TEMPLATE_URL: the URL of the instance template that you want to use to create instances in the MIG. The URL can contain either the ID or name of the instance template. Specify one of the following values:
- For a regional instance template: projects/PROJECT_ID/regions/REGION/instanceTemplates/INSTANCE_TEMPLATE_ID
- For a global instance template: INSTANCE_TEMPLATE_ID
ZONE: the zone where the MIG creates VMs.
LIST_OF_ZONES: a comma separated list of zones where the regional MIG creates VMs. For example, us-central1-a, us-central1-b, us-central1-c.

REST

To create a MIG that has bulk creation enabled, make a POST request as follows. In the request body, set the targetSizePolicy.mode field to bulk.

If the instance template that you use specifies the flex-start provisioning model, then you must turn off repairs in the MIG by including the "instanceLifecyclePolicy": {"defaultActionOnFailure": "DO_NOTHING"} field in the following requests.

To create a zonal MIG, make a POST request to the instanceGroupManagers.insert method.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers

{
  "name": "INSTANCE_GROUP_NAME",
  "targetSize": SIZE,
  "targetSizePolicy": {
    "mode": "bulk"
  },
  "instanceLifecyclePolicy": {
      "defaultActionOnFailure": "DO_NOTHING"
  },
  "versions": [
    {
      "instanceTemplate": "INSTANCE_TEMPLATE_URL"
    }
  ]
}

To create a regional MIG, make a POST request to the regionInstanceGroupManagers.insert method.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instanceGroupManagers

{
  "name": "INSTANCE_GROUP_NAME",
  "targetSize": SIZE,
  "targetSizePolicy": {
    "mode": "bulk"
  },
  "instanceLifecyclePolicy": {
      "defaultActionOnFailure": "DO_NOTHING"
  },
  "versions": [
    {
      "instanceTemplate": "INSTANCE_TEMPLATE_URL"
    }
  ]
}

Replace the following:

PROJECT_ID:the project ID.
ZONE:the zone where the MIG creates VMs.
REGION:the region where the MIG creates VMs.
INSTANCE_GROUP_NAME:the name of the MIG.
SIZE:the number of VMs in the MIG.
INSTANCE_TEMPLATE_URL: the URL of the instance template that you want to use to create instances in the MIG. The URL can contain either the ID or name of the instance template. Specify one of the following values:
- For a regional instance template: projects/PROJECT_ID/regions/REGION/instanceTemplates/INSTANCE_TEMPLATE_ID
- For a global instance template: INSTANCE_TEMPLATE_ID

What's next

Learn how to view information about MIGs and managed VMs.

Learn how to view the actual and forecasted usage of your VMs and GPUs.

Create a MIG that adds GPU VMs all at once Stay organized with collections Save and categorize content based on your preferences.

Before you begin

Console

gcloud

REST

Limitations

Create a MIG and add GPU VMs all at once

Create an instance template

Permissions required for this task

Console

gcloud

REST

Create a MIG and add GPU VMs all at once

Permissions required for this task

Console

gcloud

REST

What's next

Create a MIG that adds GPU VMs all at once