Create a TPU VM instance

This document describes how to create a TPU virtual machine (VM) instance. A TPU VM is also called a single-host TPU slice.

To create a group of single-host slices, see Create a MIG with single-host TPU slices.

Before you begin

• If you haven't already, set up authentication. Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

  Select the tab for how you plan to use the samples on this page:

  gcloud

  1. Install the Google Cloud CLI. After installation, initialize the Google Cloud CLI by running the following command:

     gcloud init

     If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

     Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.
  2. Set a default region and zone.

  REST

  To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

  Install the Google Cloud CLI.

  If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

  For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

Prerequisites

Before you proceed to create a TPU instance, you must do the following:

1. Choose the TPU version: select the TPU version that is suitable for your workload. For a list of TPU versions by workload type, see Recommended TPU versions by workload types.

2. Validate TPU availability in your preferred location: TPUs are available in specific Google Cloud regions. To use a TPU version, ensure its availability in your preferred region. For a list of TPU locations, see TPU availability.

3. Ensure that your project has sufficient TPU quota: if you are creating a TPU instance with on-demand or Spot VMs, you must have sufficient TPU quota available in the region that you want to use. Creating a TPU instance that consumes a TPU reservation doesn't require any TPU quota as quota is used when the reservation was created. For a list of TPU quota names, see TPU quota and for instructions on how to view the quota, see View and manage quotas

4. Choose a TPU consumption option: select a consumption option that best fits your workload, its duration, and your cost needs. For a list of consumption option availability by TPU versions, see TPU consumption options.

Create a TPU VM instance

The parameters you use to create a TPU VM instance depend on the consumption option you are using: on-demand, Spot, reservation-bound, or flex-start. For more information, see About VM provisioning models .

Create an on-demand TPU VM instance

To create an on-demand TPU VM instance, use the gcloud compute instances create command:

Important: When you use the ct6e-standard-8t machine type, you must set the threads per core to 1 by including the --threads-per-core=1 flag in the following command. For more information on threads per core, see Set the number of threads per core.

gcloud

gcloud compute instances create TPU_NAME \
  --machine-type=MACHINE_TYPE \
  --image-family=IMAGE_FAMILY \
  --image-project=IMAGE_PROJECT \
  --zone=ZONE \
  --maintenance-policy=TERMINATE

REST

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "TPU_NAME",
    "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE",
    "disks": [
      {
        "boot": true,
        "initializeParams": {
          "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
        }
      }
    ],
    "scheduling": {
      "onHostMaintenance": "TERMINATE"
    },
    "networkInterfaces": [
      {
        "network": "global/networks/default"
      }
    ]
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances"

Replace the following placeholders:

• PROJECT_ID: The ID of your Google Cloud project.
• TPU_NAME: A name for your TPU VM.
• MACHINE_TYPE: The machine type for the TPU VM (for example ct6e-standard-8t).
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.
• ZONE: The zone for the TPU VM (for example us-central1-b).

Create a TPU Spot VM instance

To create a TPU Spot VM instance, use the gcloud compute instances create command with the --provisioning-model=SPOT flag:

Important: When you use the ct6e-standard-8t machine type, you must set the threads per core to 1 by including the --threads-per-core=1 flag in the following command. For more information on threads per core, see Set the number of threads per core.

gcloud

gcloud compute instances create TPU_NAME \
  --machine-type=MACHINE_TYPE \
  --image-family=IMAGE_FAMILY \
  --image-project=IMAGE_PROJECT \
  --zone=ZONE \
  --provisioning-model=SPOT \
  --instance-termination-action=DELETE \
  --maintenance-policy=TERMINATE

REST

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "TPU_NAME",
    "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE",
    "disks": [
      {
        "boot": true,
        "initializeParams": {
          "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
        }
      }
    ],
    "scheduling": {
      "onHostMaintenance": "TERMINATE",
      "provisioningModel": "SPOT",
      "instanceTerminationAction": "DELETE"
    },
    "networkInterfaces": [
      {
        "network": "global/networks/default"
      }
    ]
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances"

Replace the following placeholders:

• PROJECT_ID: The ID of your Google Cloud project.
• TPU_NAME: A name for your TPU VM.
• MACHINE_TYPE: The machine type for the TPU VM (for example ct6e-standard-8t).
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.
• ZONE: The zone for the TPU VM (for example us-central1-b).

Create a TPU VM instance using a reservation

To create a TPU VM instance using the reservation-bound consumption option, use the gcloud compute instances create command with the --reservation-affinity=specific and --reservation flags:

Important: When you use the ct6e-standard-8t machine type, you must set the threads per core to 1 by including the --threads-per-core=1 flag in the following command. For more information on threads per core, see Set the number of threads per core.

gcloud

gcloud compute instances create TPU_NAME \
  --machine-type=MACHINE_TYPE \
  --image-family=IMAGE_FAMILY \
  --image-project=IMAGE_PROJECT \
  --zone=ZONE \
  --provisioning-model=reservation-bound \
  --reservation-affinity=specific \
  --reservation=RESERVATION_NAME \
  --instance-termination-action=DELETE \
  --maintenance-policy=TERMINATE

REST

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "TPU_NAME",
    "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE",
    "disks": [
      {
        "boot": true,
        "initializeParams": {
          "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
        }
      }
    ],
    "scheduling": {
      "onHostMaintenance": "TERMINATE",
      "provisioningModel": "RESERVATION_BOUND",
      "instanceTerminationAction": "DELETE"
    },
    "networkInterfaces": [
      {
        "network": "global/networks/default"
      }
    ],
    "reservationAffinity": {
      "consumeReservationType": "SPECIFIC_RESERVATION",
      "key": "compute.googleapis.com/reservation-name",
      "values": [
        "RESERVATION_NAME"
      ]
    }
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances"

Replace the following placeholders:

• PROJECT_ID: The ID of your Google Cloud project.
• TPU_NAME: A name for your TPU VM.
• MACHINE_TYPE: The machine type for the TPU VM (for example ct6e-standard-8t).
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.
• ZONE: The zone for the TPU VM (for example us-central1-b).
• RESERVATION_NAME: The name of your reservation.

Create a TPU Flex-start VM

To create a TPU Flex-start VM instance, use the gcloud compute instances create command with the --provisioning-model=FLEX_START flag:

Important: When you use the ct6e-standard-8t machine type, you must set the threads per core to 1 by including the --threads-per-core=1 flag in the following command. For more information on threads per core, see Set the number of threads per core.

gcloud

gcloud compute instances create TPU_NAME \
  --machine-type=MACHINE_TYPE \
  --image-family=IMAGE_FAMILY \
  --image-project=IMAGE_PROJECT \
  --zone=ZONE \
  --provisioning-model=FLEX_START \
  --max-run-duration=MAX_RUN_DURATION \
  --request-valid-for-duration=VALID_DURATION \
  --instance-termination-action=DELETE \
  --maintenance-policy=TERMINATE

REST

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "TPU_NAME",
    "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE",
    "disks": [
      {
        "boot": true,
        "initializeParams": {
          "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
        }
      }
    ],
    "params": {
      "requestValidForDuration": {
        "seconds": VALID_DURATION
      }
    },
    "scheduling": {
      "onHostMaintenance": "TERMINATE",
      "provisioningModel": "FLEX_START",
      "instanceTerminationAction": "DELETE",
      "maxRunDuration": {
        "seconds": MAX_RUN_DURATION
      }
    },
    "networkInterfaces": [
      {
        "network": "global/networks/default"
      }
    ]
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances"

Replace the following placeholders:

• PROJECT_ID: The ID of your Google Cloud project.
• TPU_NAME: A name for your TPU VM.
• MACHINE_TYPE: The machine type for the TPU VM (for example ct6e-standard-8t).
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.
• ZONE: The zone for the TPU VM (for example us-central1-b).
• MAX_RUN_DURATION: The maximum run duration for the TPU VM (for example 6h). For REST, specify this as a number of seconds (for example 21600 for 6 hours).
• VALID_DURATION: The maximum duration for which the request is valid. For REST, specify this as a number of seconds (for example 3600 for 1 hour). request is valid.

For more information about creating Compute Engine instances, see Create and start a Compute Engine instance.

Limitations

• Lifecycle operations: You can't stop, start, resume, or suspend TPU instances. To change configurations that require a restart or to stop incurring charges, you must delete the instances.

• Update VM details: You can update the properties that only require a refresh (REFRESH) action. If a configuration requires a restart (RESTART), then you must recreate the instance.

What's next

• Connect to a TPU instance
• View the details of a TPU instance
• Monitor TPUs
• Delete a TPU instance