This document describes how to create a managed instance group (MIG) that
forms a multi-host TPU slice.

> [!NOTE]
> **Note:** You can also create a managed compute cluster in Cluster Director to provision a multi-host TPU slice with an automated high-throughput workload policy, Cloud TPU runtime library installation, and shared storage mounts. For more information, see [Create a custom cluster](https://docs.cloud.google.com/cluster-director/docs/create/custom-cluster.md.txt) and [Submit and monitor jobs on managed compute clusters](https://docs.cloud.google.com/cluster-director/docs/workloads/configure-mig.md.txt) in the Cluster Director documentation.

## Before you begin

- Review the [limitations](https://docs.cloud.google.com/compute/docs/tpus/tpus-and-migs.md.txt#limitations) for creating MIGs with TPU instances.
- If you haven't already, set up [authentication](https://docs.cloud.google.com/compute/docs/authentication.md.txt). Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:
  1.
     [Install](https://docs.cloud.google.com/sdk/docs/install) the Google Cloud CLI.

     After installation,
     [initialize](https://docs.cloud.google.com/sdk/docs/initializing) the Google Cloud CLI by running the following command:

     ```bash
     gcloud init
     ```


     If you're using an external identity provider (IdP), you must first
     [sign in to the gcloud CLI with your federated identity](https://docs.cloud.google.com/iam/docs/workforce-log-in-gcloud.md.txt).

     > [!NOTE]
     > **Note:** If you installed the gcloud CLI previously, make sure you have the latest version by running `gcloud components update`.

  2. [Set a default region and zone](https://docs.cloud.google.com/compute/docs/gcloud-compute.md.txt#set_default_zone_and_region_in_your_local_client).

## Prerequisites

Before you proceed to create a multi-host TPU slice, you must do the following:

1. **Choose the TPU version** : select the TPU version that is suitable for your
   workload. For a list of TPU versions by workload type, see
   [Recommended TPU versions by workload types](https://docs.cloud.google.com/compute/docs/tpus/tpu-machines.md.txt#tpu_versions_and_workload_types).

   <br />

2. **Choose a TPU consumption option** : select a consumption option that best
   fits your workload, its duration, and your cost needs.
   For a list of
   consumption option availability by TPU versions, see
   [TPU consumption options](https://docs.cloud.google.com/compute/docs/tpus/tpu-machines.md.txt#consumption_options).


3. **Validate TPU availability in your preferred location** : TPUs are available
   in specific Google Cloud regions. To use a TPU version, ensure its
   availability in your preferred region. For a list of TPU locations, see
   [TPU availability](https://docs.cloud.google.com/compute/docs/regions-zones/tpu-regions-zones.md.txt#view-using-table).

4. **Ensure that your project has sufficient TPU quota** : if you are creating
   a multi-host TPU slice with
   on-demand, Flex-start VMs,
   or Spot VMs, you must have sufficient TPU quota available in the
   region that you want to use. Creating a multi-host TPU slice that consumes a TPU
   reservation doesn't require any TPU quota as quota is used when the
   reservation was created. For a list of TPU quota names, see
   [TPU quota](https://docs.cloud.google.com/compute/resource-usage.md.txt#tpu_quota) and for instructions on how
   to view the quota, see
   [View and manage quotas](https://docs.cloud.google.com/docs/quotas/view-manage.md.txt)

5. **Choose a topology** : select a topology supported for the selected TPU
   version. For a list of topologies available for each TPU version, see
   [TPU topology](https://docs.cloud.google.com/compute/docs/tpus/tpu-machines.md.txt#tpu_topology).

## Create a MIG with multi-host TPU slices

1. Create an instance template.
2. Create a workload policy.
3. Create the MIG.

### Create an instance template

The command to create an instance template depends on the consumption
option you use: on-demand, Spot, reservation-bound, or flex-start. For more
information about consumption options, see [About VM provisioning models](https://docs.cloud.google.com/compute/docs/instances/provisioning-models.md.txt).

#### Create an instance template for an on-demand TPU VM

The following command creates an instance template using the on-demand
consumption option:

### gcloud

```bash
gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
  --machine-type=MACHINE_TYPE \
  --maintenance-policy=TERMINATE \
  --image-family=IMAGE_FAMILY \
  --image-project=IMAGE_PROJECT
```

### REST

```bash
curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "INSTANCE_TEMPLATE_NAME",
    "properties": {
      "machineType": "MACHINE_TYPE",
      "scheduling": {
        "onHostMaintenance": "TERMINATE"
      },
      "disks": [
        {
          "boot": true,
          "initializeParams": {
            "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
          }
        }
      ],
      "networkInterfaces": [
        {
          "network": "global/networks/default"
        }
      ]
    }
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instanceTemplates"
```

Replace the following placeholders:

- `PROJECT_ID`: The ID of your Google Cloud project.
- `INSTANCE_TEMPLATE_NAME`: The name of your instance template.
- `MACHINE_TYPE`: The [machine type](https://docs.cloud.google.com/compute/docs/tpus/tpu-machines.md.txt#tpu_machine_types) for the TPU VM, for example, `ct6e-standard-8t`.
- `IMAGE_FAMILY`: The [OS image family](https://docs.cloud.google.com/compute/docs/images/os-details.md.txt#ga_ubuntu_lts_versions) for the TPU VM. If you want to install a specific OS version, use the `--image` flag. For more information about OS images, see [OS images](https://docs.cloud.google.com/compute/docs/images.md.txt).
- `IMAGE_PROJECT`: The project that contains the OS image. For TPU images, this is `ubuntu-os-accelerator-images`.

#### Create an instance template for a TPU Spot VM

The following command creates an instance template using the Spot consumption option:

### gcloud

```bash
gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
  --machine-type=MACHINE_TYPE \
  --maintenance-policy=TERMINATE \
  --instance-termination-action=STOP \
  --provisioning-model=SPOT \
  --image-family=IMAGE_FAMILY \
  --image-project=IMAGE_PROJECT
```

### REST

```bash
curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "INSTANCE_TEMPLATE_NAME",
    "properties": {
      "machineType": "MACHINE_TYPE",
      "disks": [
        {
          "boot": true,
          "initializeParams": {
            "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
          }
        }
      ],
      "scheduling": {
        "onHostMaintenance": "TERMINATE",
        "provisioningModel": "SPOT",
        "instanceTerminationAction": "STOP"
      },
      "networkInterfaces": [
        {
          "network": "global/networks/default"
        }
      ]
    }
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instanceTemplates"
```

Replace the following placeholders:

- `PROJECT_ID`: The ID of your Google Cloud project.
- `INSTANCE_TEMPLATE_NAME`: The name of your instance template.
- `MACHINE_TYPE`: The [machine type](https://docs.cloud.google.com/compute/docs/tpus/tpu-machines.md.txt#tpu_machine_types) for the TPU VM, for example, `ct6e-standard-8t`.
- `IMAGE_FAMILY`: The [OS image family](https://docs.cloud.google.com/compute/docs/images/os-details.md.txt#ga_ubuntu_lts_versions) for the TPU VM. If you want to install a specific OS version, use the `--image` flag. For more information about OS images, see [OS images](https://docs.cloud.google.com/compute/docs/images.md.txt).
- `IMAGE_PROJECT`: The project that contains the OS image. For TPU images, this is `ubuntu-os-accelerator-images`.

#### Create an instance template for a TPU reservation-bound VM

The following command creates an instance template using the reservation-bound
consumption option:

### gcloud

```bash
gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
  --machine-type=MACHINE_TYPE \
  --maintenance-policy=TERMINATE \
  --instance-termination-action=DELETE \
  --reservation-affinity=specific \
  --provisioning-model=reservation-bound \
  --reservation=RESERVATION_NAME \
  --image-family=IMAGE_FAMILY \
  --image-project=IMAGE_PROJECT
```

### REST

```bash
curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "INSTANCE_TEMPLATE_NAME",
    "properties": {
      "machineType": "MACHINE_TYPE",
      "disks": [
        {
          "boot": true,
          "initializeParams": {
            "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
          }
        }
      ],
      "scheduling": {
        "onHostMaintenance": "TERMINATE",
        "provisioningModel": "RESERVATION_BOUND",
        "instanceTerminationAction": "DELETE"
      },
      "reservationAffinity": {
        "consumeReservationType": "SPECIFIC_RESERVATION",
        "key": "compute.googleapis.com/reservation-name",
        "values": [
          "RESERVATION_NAME"
        ]
      },
      "networkInterfaces": [
        {
          "network": "global/networks/default"
        }
      ]
    }
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instanceTemplates"
```

Replace the following placeholders:

- `PROJECT_ID`: The ID of your Google Cloud project.
- `INSTANCE_TEMPLATE_NAME`: The name of your instance template.
- `MACHINE_TYPE`: The [machine type](https://docs.cloud.google.com/compute/docs/tpus/tpu-machines.md.txt#tpu_machine_types) for the TPU VM, for example, `ct6e-standard-8t`.
- `RESERVATION_NAME`: The name of the specific
  reservation to consume. For All Capacity mode reservations, you can target
  a block or sub-block by specifying the reservation URL as follows:

  - Specify a block:

    ```bash
    RESERVATION_NAME/reservationBlocks/RESERVATION_BLOCK_NAME
    ```
  - Specify a sub-block:

    ```bash
    RESERVATION_NAME/reservationBlocks/RESERVATION_BLOCK_NAME/reservationSubBlocks/RESERVATION_SUBBLOCK_NAME
    ```
- `IMAGE_FAMILY`: The [OS image family](https://docs.cloud.google.com/compute/docs/images/os-details.md.txt#ga_ubuntu_lts_versions)
  for the TPU VM. If you want to install a specific OS version, use the
  `--image` flag. For more information about OS images, see [OS
  images](https://docs.cloud.google.com/compute/docs/images).

- `IMAGE_PROJECT`: The project that contains the OS
  image. For TPU images, this is `ubuntu-os-accelerator-images`.

#### Create an instance template for a TPU Flex-start VM

The following command creates an instance template using the flex-start consumption option:

### gcloud

```bash
gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --machine-type=MACHINE_TYPE \
    --maintenance-policy=TERMINATE \
    --instance-termination-action=DELETE \
    --provisioning-model=FLEX_START \
    --max-run-duration=DURATION \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT
```

### REST

```bash
curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "INSTANCE_TEMPLATE_NAME",
    "properties": {
      "machineType": "MACHINE_TYPE",
      "disks": [
        {
          "boot": true,
          "initializeParams": {
            "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
          }
        }
      ],
      "scheduling": {
        "onHostMaintenance": "TERMINATE",
        "provisioningModel": "FLEX_START",
        "instanceTerminationAction": "DELETE",
        "maxRunDuration": {
          "seconds": "DURATION"
        }
      },
      "networkInterfaces": [
        {
          "network": "global/networks/default"
        }
      ]
    }
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instanceTemplates"
```

Replace the following placeholders:

- `PROJECT_ID`: The ID of your Google Cloud project.
- `INSTANCE_TEMPLATE_NAME`: The name of your instance template.
- `MACHINE_TYPE`: The [machine type](https://docs.cloud.google.com/compute/docs/tpus/tpu-machines.md.txt#tpu_machine_types) for the TPU VM, for example, `ct6e-standard-8t`.
- `DURATION`: The maximum duration for which the TPU VM can run. For REST, specify this as a number of seconds (for example, `3600` for 1 hour, or `604800` for 7 days).
- `IMAGE_FAMILY`: The [OS image family](https://docs.cloud.google.com/compute/docs/images/os-details.md.txt#ga_ubuntu_lts_versions) for the TPU VM. If you want to install a specific OS version, use the `--image` flag. For more information about OS images, see [OS images](https://docs.cloud.google.com/compute/docs/images.md.txt).
- `IMAGE_PROJECT`: The project that contains the OS image. For TPU images, this is `ubuntu-os-accelerator-images`.

### Create a workload policy

You must create a workload policy with the `accelerator-topology` parameter (for
example, `4x4`, `8x8`, or `4x4x4`). The accelerator topology configures the MIG
to treat the instances as a single, interconnected slice.

The following command creates a workload policy for a specific topology:

### gcloud

```bash
gcloud compute resource-policies create workload-policy WORKLOAD_POLICY_NAME \
  --type=high-throughput \
  --accelerator-topology=TOPOLOGY \
  --region=REGION
```

### REST

```bash
curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "WORKLOAD_POLICY_NAME",
    "workloadPolicy": {
      "type": "HIGH_THROUGHPUT",
      "acceleratorTopology": "TOPOLOGY"
    }
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/resourcePolicies"
```

Replace the following placeholders:

- `PROJECT_ID`: The ID of your Google Cloud project.
- `WORKLOAD_POLICY_NAME`: The name of your workload policy.
- `TOPOLOGY`: The topology of the TPU VMs, for example, `4x4x8`. For more information about topology for each version of TPU, see [TPU topology](https://docs.cloud.google.com/compute/docs/tpus/tpu-machines.md.txt#tpu_topology).
- `REGION`: The [region](https://docs.cloud.google.com/compute/docs/regions-zones/tpu-regions-zones.md.txt) for your workload policy.

### Create a MIG

To create a zonal MIG containing a multi-host TPU slice, use the following
command:

### gcloud

```bash
gcloud compute instance-groups managed create MIG_NAME \
   --size=MIG_SIZE \
   --target-size-policy-mode=bulk \
   --template=INSTANCE_TEMPLATE_URL \
   --zone=ZONE \
   --default-action-on-vm-failure=do-nothing \
   --workload-policy=WORKLOAD_POLICY_URL
```

### REST

```bash
curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "MIG_NAME",
    "targetSize": MIG_SIZE,
    "targetSizePolicy": {
      "mode": "BULK"
    },
    "instanceTemplate": "INSTANCE_TEMPLATE_URL",
    "instanceLifecyclePolicy": {
      "defaultActionOnFailure": "DO_NOTHING"
    },
    "resourcePolicies": {
      "workloadPolicy": "WORKLOAD_POLICY_URL"
    }
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers"
```

Replace the following placeholders:

- `PROJECT_ID`: The ID of your Google Cloud project.
- `MIG_NAME`: The name of your MIG.
- `MIG_SIZE`: The size of the MIG.
- `INSTANCE_TEMPLATE_URL`: the URL of the instance template that you want to use to create instances in the MIG. The URL can contain either the [ID](https://docs.cloud.google.com/compute/docs/instance-templates/get-list-delete-instance-templates.md.txt#get_information_about_an_instance_template) or name of the instance template. Specify one of the following values:
  - For a regional instance template: `projects/PROJECT_ID/regions/REGION/instanceTemplates/INSTANCE_TEMPLATE_ID`
  - For a global instance template: `INSTANCE_TEMPLATE_ID`
- `ZONE`: The [zone](https://docs.cloud.google.com/compute/docs/regions-zones/tpu-regions-zones.md.txt) for your MIG.
- `WORKLOAD_POLICY_URL`: The URL of the workload policy that you want to use to create instances in the MIG. For example: `projects/PROJECT_ID/regions/WORKLOAD_POLICY_REGION/resourcePolicies/WORKLOAD_POLICY_NAME`.

To create a regional MIG containing a multi-host TPU slice, use the
following command:

### gcloud

```bash
gcloud compute instance-groups managed create MIG_NAME \
   --size=MIG_SIZE \
   --target-size-policy-mode=bulk \
   --template=INSTANCE_TEMPLATE_URL \
   --region=REGION \
   --default-action-on-vm-failure=do-nothing \
   --workload-policy=WORKLOAD_POLICY_URL \
   --target-distribution-shape=any-single-zone \
   --instance-redistribution-type=none
```

### REST

```bash
curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "MIG_NAME",
    "targetSize": MIG_SIZE,
    "targetSizePolicy": {
      "mode": "BULK"
    },
    "instanceTemplate": "INSTANCE_TEMPLATE_URL",
    "instanceLifecyclePolicy": {
      "defaultActionOnFailure": "DO_NOTHING"
    },
    "resourcePolicies": {
      "workloadPolicy": "WORKLOAD_POLICY_URL"
    },
    "distributionPolicy": {
      "targetShape": "ANY_SINGLE_ZONE"
    },
    "updatePolicy": {
      "instanceRedistributionType": "NONE"
    }
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instanceGroupManagers"
```

Replace the following placeholders:

- `PROJECT_ID`: The ID of your Google Cloud project.
- `MIG_NAME`: The name of your MIG.
- `MIG_SIZE`: The number of VMs in the MIG.
- `INSTANCE_TEMPLATE_URL`: the URL of the instance template that you want to use to create instances in the MIG. The URL can contain either the [ID](https://docs.cloud.google.com/compute/docs/instance-templates/get-list-delete-instance-templates.md.txt#get_information_about_an_instance_template) or name of the instance template. Specify one of the following values:
  - For a regional instance template: `projects/PROJECT_ID/regions/REGION/instanceTemplates/INSTANCE_TEMPLATE_ID`
  - For a global instance template: `INSTANCE_TEMPLATE_ID`
- `ZONE`: The [zone](https://docs.cloud.google.com/compute/docs/regions-zones/tpu-regions-zones.md.txt) for your MIG.
- `REGION`: The [region](https://docs.cloud.google.com/compute/docs/regions-zones/tpu-regions-zones.md.txt) for your MIG.
- `WORKLOAD_POLICY_URL`: The URL of the workload policy that you want to use to create instances in the MIG. For example: `projects/PROJECT_ID/regions/WORKLOAD_POLICY_REGION/resourcePolicies/WORKLOAD_POLICY_NAME`.

## Create VMs with custom names in a MIG


You can create VMs in a MIG by specifying custom names for each VM. This is
useful for debugging and ensuring instances are created in a specific order.

MIGs that contain a multi-host TPU slice use the bulk mode of target size
policy. When creating VMs with custom names in such a MIG, the following
applies:

- You must first
  [verify](https://docs.cloud.google.com/compute/docs/instance-groups/getting-info-about-migs.md.txt#listing_a_migs_managed_instances)
  that the MIG doesn't have VMs in it. If the MIG has VMs, you must either
  [resize the MIG](https://docs.cloud.google.com/compute/docs/instance-groups/add-remove-vms-in-mig.md.txt#resize_managed_group)
  to target size `0` or [create another MIG](https://docs.cloud.google.com/compute/docs/tpus/create-tpu-mig-multi-host-slice.md.txt#create-mig) with target size `0`.

- You can only use the REST API to create VMs with custom names.

<br />


Create VMs with custom names by using one of the following REST API
methods:

- For a zonal MIG, use the [`instanceGroupManagers.createInstances`](https://docs.cloud.google.com/compute/docs/reference/rest/v1/instanceGroupManagers/createInstances.md.txt).

  <br />

  ```
   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/MIG_NAME/createInstances
   {
     "instances": [
       {
         "name": "INSTANCE_NAME_1"
       },
       {
         "name": "INSTANCE_NAME_2"
       },
       ...
     ]
   }
   
  ```

  <br />

- For a regional MIG, use the [`regionInstanceGroupManagers.createInstances`](https://docs.cloud.google.com/compute/docs/reference/rest/v1/regionInstanceGroupManagers/createInstances.md.txt).

  <br />

  ```
   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instanceGroupManagers/MIG_NAME/createInstances
   {
     "instances": [
       {
         "name": "INSTANCE_NAME_1"
       },
       {
         "name": "INSTANCE_NAME_2"
       },
       ...
     ]
   }
   
  ```

  <br />

Replace the following placeholders:

- `PROJECT_ID`: The ID of the project where the MIG exists.
- `ZONE`: The [zone](https://docs.cloud.google.com/compute/docs/regions-zones/tpu-regions-zones.md.txt) of the MIG.
- `REGION`: The [region](https://docs.cloud.google.com/compute/docs/regions-zones/tpu-regions-zones.md.txt) of the MIG.
- `INSTANCE_NAME_1,2,..`: The names of the VMs to add to the specified MIG.

<br />

## What's next

- [Connect to a TPU instance](https://docs.cloud.google.com/compute/docs/connect/standard-ssh.md.txt)
- [View a MIG's properties](https://docs.cloud.google.com/compute/docs/instance-groups/getting-info-about-migs.md.txt#getting_a_migs_properties)
- [View the status of a TPU slice](https://docs.cloud.google.com/compute/docs/instance-groups/getting-info-about-migs.md.txt#view_the_status_of_a_tpu_slice)
- [Monitor TPUs](https://docs.cloud.google.com/tpu/docs/monitor-tpu.md.txt)
- [Delete a MIG](https://docs.cloud.google.com/compute/docs/instance-groups/delete-mig.md.txt)
- [Create a custom cluster](https://docs.cloud.google.com/cluster-director/docs/create/custom-cluster.md.txt)
- [Submit and monitor jobs on managed compute clusters](https://docs.cloud.google.com/cluster-director/docs/workloads/configure-mig.md.txt)