Create a MIG with single-host Cloud TPU slices

This document describes how to create a managed instance group (MIG) with multiple, independent single-host TPU slices.

Prerequisites

Complete the following prerequisites:

1. Create a project for your TPUs as described in Set up a project for TPUs.
2. Determine your TPU requirements as described in Plan your resources.

Create a MIG with multiple single-host TPU slices

To create multiple, independent TPU VMs in a MIG:

1. Create an instance template.
2. Create the MIG.

Create an instance template

Instance template configurations vary based on the consumption option you use: on-demand, Spot, reservations, or Flex-start. For more information about consumption options, see Plan your TPU resources .

Create an instance template for an on-demand TPU VM

The following command creates an instance template that uses the on-demand consumption option:

Important: When you use the ct6e-standard-8t machine type, you must set the threads per core to 1 by including the --threads-per-core=1 flag in the following command. For more information on threads per core, see Set the number of threads per core.

gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --machine-type=MACHINE_TYPE \
    --maintenance-policy=TERMINATE \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT

Replace the following placeholders:

• INSTANCE_TEMPLATE_NAME: The name of your instance template.
• MACHINE_TYPE: The machine type for the TPU VM (for example, ct6e-standard-8t).
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.

Create an instance template for a TPU Spot VM

The following command creates an instance template that uses the Spot consumption option:

Important: When you use the ct6e-standard-8t machine type, you must set the threads per core to 1 by including the --threads-per-core=1 flag in the following command. For more information on threads per core, see Set the number of threads per core.

 gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --machine-type=MACHINE_TYPE \
    --maintenance-policy=TERMINATE \
    --instance-termination-action=STOP \
    --provisioning-model=SPOT \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT

Replace the following placeholders:

• INSTANCE_TEMPLATE_NAME: The name of your instance template.
• MACHINE_TYPE: The machine type for the TPU VM (for example, ct6e-standard-8t).
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.

Create an instance template for a TPU reservation-bound VM

The following command creates an instance template that uses the reservation-bound consumption option:

Important: When you use the ct6e-standard-8t machine type, you must set the threads per core to 1 by including the --threads-per-core=1 flag in the following command. For more information on threads per core, see Set the number of threads per core.

 gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --machine-type=MACHINE_TYPE \
    --maintenance-policy=TERMINATE \
    --instance-termination-action=DELETE \
    --reservation-affinity=specific \
    --provisioning-model=reservation-bound \
    --reservation=RESERVATION_NAME \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT

Replace the following placeholders:

• INSTANCE_TEMPLATE_NAME: The name of your instance template.
• MACHINE_TYPE: The machine type for the TPU VM (for example, ct6e-standard-8t).
• RESERVATION_NAME: The name of a specific reservation that you want to consume.
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.

Create an instance template for a TPU Flex-start VM

The following command creates an instance template that uses the Flex-start consumption option:

Important: When you use the ct6e-standard-8t machine type, you must set the threads per core to 1 by including the --threads-per-core=1 flag in the following command. For more information on threads per core, see Set the number of threads per core.

 gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --machine-type=MACHINE_TYPE \
    --maintenance-policy=TERMINATE \
    --instance-termination-action=DELETE \
    --provisioning-model=FLEX_START \
    --max-run-duration=DURATION \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT

Replace the following placeholders:

• INSTANCE_TEMPLATE_NAME: The name of your instance template.
• MACHINE_TYPE: The machine type for the TPU VM (for example, ct6e-standard-8t).
• DURATION: The maximum run duration for the TPU VM (for example, 1h).
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.

Create a MIG

Create a zonal or a regional MIG by using the gcloud compute instance-groups managed create command as follows:

• To create a zonal MIG containing a single-host TPU slice, use the following command:

  gcloud compute instance-groups managed create MIG_NAME \
      --size=MIG_SIZE \
      --template=INSTANCE_TEMPLATE_URL \
      --zone=ZONE
  

• To create a regional MIG containing a single-host TPU slice, use the following command:

  gcloud compute instance-groups managed create MIG_NAME \
      --size=MIG_SIZE \
      --template=INSTANCE_TEMPLATE_URL \
      --region=REGION \
      --target-distribution-shape=ANY_SINGLE_ZONE \
      --instance-redistribution-type=none
  

Replace the following placeholders:

• MIG_NAME: The name of your MIG.
• MIG_SIZE: The number of VMs in the MIG.
• INSTANCE_TEMPLATE_URL: the URL of the instance template that you want to use to create instances in the MIG. The URL can contain either the ID or name of the instance template. Specify one of the following values:
  • For a regional instance template: projects/PROJECT_ID/regions/REGION/instanceTemplates/INSTANCE_TEMPLATE_ID
  • For a global instance template: INSTANCE_TEMPLATE_ID
• ZONE: The zone where you want your MIG to be provisioned.
• REGION: The region where you want your MIG to be provisioned.

Create VMs with custom names in a MIG

You can create VMs in a MIG by specifying custom names for each VM. This is useful for debugging and ensuring instances are created in a specific order.

gcloud

Create a VM with custom name by using the create-instance command and specify the --instance flag. To create multiple VMs, repeat the command for each VM. Otherwise, use the REST API method to specify multiple VMs in a single request.

• For a zonal MIG, use the following command:

  gcloud compute instance-groups managed create-instance MIG_NAME \
      --zone=ZONE \
      --instance=INSTANCE_NAME
  

• For a regional MIG, use the following command:

  gcloud compute instance-groups managed create-instance MIG_NAME \
      --region=REGION \
      --instance=INSTANCE_NAME
  

Replace the following placeholders:

• MIG_NAME: The name of your MIG.
• ZONE: The zone of the MIG.
• REGION: The region of the MIG.
• INSTANCE_NAME: The name of the VM to add to the specified MIG.

REST

Create VMs with custom names by using one of the following REST API methods:

• For a zonal MIG, use the instanceGroupManagers.createInstances.

   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/MIG_NAME/createInstances
   {
     "instances": [
       {
         "name": "INSTANCE_NAME_1"
       },
       {
         "name": "INSTANCE_NAME_2"
       },
       ...
     ]
   }
   

• For a regional MIG, use the regionInstanceGroupManagers.createInstances.

   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instanceGroupManagers/MIG_NAME/createInstances
   {
     "instances": [
       {
         "name": "INSTANCE_NAME_1"
       },
       {
         "name": "INSTANCE_NAME_2"
       },
       ...
     ]
   }
   

Replace the following placeholders:

• PROJECT_ID: The ID of the project where the MIG exists.
• ZONE: The zone of the MIG.
• REGION: The region of the MIG.
• INSTANCE_NAME_1,2,..: The names of the VMs to add to the specified MIG.

What's next

• Learn about TPU VMs and MIGs.
• Learn how to Create a MIG with a multi-host Cloud TPU slice.
• Learn how to manage TPU VMs.
• Learn about TPUs in GKE.
• Learn how to run an ML workload on TPUs, for example, Serve Qwen2-72B-Instruct with vLLM on TPUs.