Share a Cloud TPU reservation

This document describes how to share a TPU reservation across multiple projects. Reservation sharing lets you create a Cloud TPU reservation in one project (the owner project) and use that reservation in another project (a consumer project) within the same organization as the owner project. Reservation sharing also lets you run custom training or prediction jobs in Vertex AI using a TPU reservation.

The owner project is the project where you created the reservation. Consumer projects are one or more projects that share the reservation. You can create TPU VMs in the owner project and consumer projects using the reservation in the owner project.

For more information about obtaining a TPU reservation, see About Cloud TPU reservations.

Limitations

Before you modify a TPU reservation, consider the following:

  • You can modify a shared reservation only from its owner project.

  • You can modify a reservation only as follows:

Convert a single-project reservation into a shared reservation

You can convert a single-project reservation into a shared reservation. Modifying the share type of a TPU reservation helps you share reserved resources with up to 100 projects in your organization. All projects that share a TPU reservation must be part of the same organization.

Before you convert a single-project reservation into a shared reservation, verify that you have sufficient quota. There are different quota requirements for shared and single-project reservations. For more information, see Additional quota requirements for shared reservations.

To enable sharing of a TPU reservation, do the following:

  1. Grant permission to the owner project to create and modify shared reservations using the gcloud resource-manager org-policies allow command:

    gcloud resource-manager org-policies allow \
        compute.sharedReservationsOwnerProjects \
        projects/OWNER_PROJECT_NUMBER --organization=ORGANIZATION_ID

    Replace the following:

    • OWNER_PROJECT_NUMBER: The project number (not project ID) of a project in your organization that you want to allow to create and modify shared reservations.
    • ORGANIZATION_ID: The organization resource ID of your organization.
  2. Enable CUD sharing by updating the CUD scope for your Cloud Billing. For instructions, see Enable CUD sharing for resource-based commitments.

  3. To convert a single-project reservation into a shared reservation, use the gcloud compute reservations update command with the --share-setting=projects and --add-share-with flags:

    gcloud compute reservations update RESERVATION_NAME \
        --project=OWNER_PROJECT_ID \
        --share-setting=projects \
        --add-share-with=CONSUMER_PROJECT_IDS \
        --zone=ZONE

    Replace the following:

    • RESERVATION_NAME: The name of the reservation.
    • OWNER_PROJECT_ID: The ID of the project that contains the reservation.
    • CONSUMER_PROJECT_IDS: A comma-separated list of IDs of projects that can consume this reservation. For example, project-1,project-2. You can include up to 100 consumer projects. These projects must be in the same organization as the owner project. Don't include the owner project. By default, the owner project is already allowed to consume the reservation.
    • ZONE: The zone where the reservation is located.

Convert a shared reservation into a single-project reservation

To convert a specifically targeted reservation from a shared reservation into a single-project reservation, you must stop or delete any instances that are consuming the reservation in each project that the reservation is shared with. This requirement doesn't apply to the instances in the owner project.

To stop sharing a TPU reservation with all consumer projects, change the shared reservation to a single-project reservation using the gcloud compute reservations update command with the --share-setting=local flag:

gcloud compute reservations update RESERVATION_NAME \
    --project=OWNER_PROJECT_ID \
    --share-setting=local \
    --zone=ZONE

Replace the following:

  • RESERVATION_NAME: The name of the reservation.
  • OWNER_PROJECT_ID: The ID of the project that contains the reservation.
  • ZONE: The zone where the reservation is located.

Modify the consumer projects in a shared reservation

To avoid errors when you modify the consumer projects for a shared reservation, consider the following:

  • Removal of projects: you can remove consumer projects from a shared reservation only if those projects are not using the reserved resources. To stop instances from consuming a reservation, stop or delete the instances.

  • Addition of projects: you can allow consumer projects to consume a shared reservation only if these projects are in the same organization as the owner project. To migrate a project to the owner project's organization, see Migrating projects between organization resources in the Resource Manager documentation.

Add projects to a shared reservation

To add one or more projects to a shared reservation, use the gcloud compute reservations update command with the --add-share-with flag:

gcloud compute reservations update RESERVATION_NAME \
    --project=OWNER_PROJECT_ID \
    --add-share-with=CONSUMER_PROJECT_IDS \
    --zone=ZONE

Replace the following:

  • RESERVATION_NAME: The name of the reservation.
  • OWNER_PROJECT_ID: The ID of the project that contains the reservation.
  • CONSUMER_PROJECT_IDS: A comma-separated list of project IDs with which to share the reservation. For example, project-1,project-2.
  • ZONE: The zone where the reservation is located.

Remove projects from a shared reservation

To remove one or more projects from a shared reservation, use the gcloud compute reservations update command with the --remove-share-with flag:

gcloud compute reservations update RESERVATION_NAME \
    --project=OWNER_PROJECT_ID \
    --remove-share-with=CONSUMER_PROJECT_IDS \
    --zone=ZONE

Replace the following:

  • RESERVATION_NAME: The name of the reservation.
  • OWNER_PROJECT_ID: The ID of the project that contains the reservation.
  • CONSUMER_PROJECT_IDS: A comma-separated list of project IDs that you want to stop sharing the reservation with. For example, project-1,project-2.
  • ZONE: The zone where the reservation is located.

Replace the list of projects that can consume a shared reservation

To replace the list of projects that can consume a shared reservation, use the gcloud beta compute reservations update command with the --share-with flag:

gcloud beta compute reservations update RESERVATION_NAME \
    --project=OWNER_PROJECT_ID \
    --share-with=CONSUMER_PROJECT_IDS \
    --zone=ZONE

Replace the following:

  • RESERVATION_NAME: The name of the reservation.
  • OWNER_PROJECT_ID: The ID of the project that contains the reservation.
  • CONSUMER_PROJECT_IDS: A comma-separated list of project IDs with which to share the reservation. For example, project-1,project-2.
  • ZONE: The zone where the reservation is located.

Allow Vertex AI to use a TPU reservation

You can allow training or prediction jobs in Vertex AI to consume a TPU reservation only after its start time.

To allow Vertex AI to consume a TPU reservation, use the gcloud compute reservations update command with the --reservation-sharing-policy=ALLOW_ALL flag:

gcloud compute reservations update RESERVATION_NAME \
    --project=OWNER_PROJECT_ID \
    --zone=ZONE \
    --reservation-sharing-policy=ALLOW_ALL

Replace the following:

  • RESERVATION_NAME: The name of the reservation.
  • OWNER_PROJECT_ID: The ID of the project that contains the reservation.
  • ZONE: The zone of the reservation.

Disallow Vertex AI from using a TPU reservation

Before you disallow Vertex AI from consuming a TPU reservation, stop any Vertex AI jobs that use it. Otherwise, you encounter errors.

To disallow Vertex AI from consuming a TPU reservation, use the gcloud compute reservations update command with the --reservation-sharing-policy=DISALLOW_ALL flag:

gcloud compute reservations update RESERVATION_NAME \
    --project=OWNER_PROJECT_ID \
    --zone=ZONE \
    --reservation-sharing-policy=DISALLOW_ALL

Replace the following:

  • RESERVATION_NAME: The name of the reservation.
  • OWNER_PROJECT_ID: The ID of the project that contains the reservation.
  • ZONE: The zone of the reservation.

Get information about a shared reservation

You can get information about a shared reservation only from its owner project.

To get information about a shared reservation, use the gcloud compute reservations describe command:

gcloud compute reservations describe RESERVATION_NAME \
    --project=OWNER_PROJECT_ID \
    --zone=ZONE

Replace the following:

  • RESERVATION_NAME: The name of the reservation.
  • OWNER_PROJECT_ID: The ID of the project that contains the reservation.
  • ZONE: The zone of the reservation.

The response is similar to the following:

aggregateReservation:
[...]
name: RESERVATION_NAME
reservationSharingPolicy:
  serviceShareType: ALLOW_ALL
resourceStatus: {}
selfLink: https://www.googleapis.com/compute/v1/projects/OWNER_PROJECT_ID/zones/ZONE/reservations/RESERVATION_NAME
shareSettings:
  projectMap:
    'CONSUMER_PROJECT_1':
      projectId: 'CONSUMER_PROJECT_1'
    'CONSUMER_PROJECT_2':
      projectId: 'CONSUMER_PROJECT_2'
  shareType: SPECIFIC_PROJECTS
specificReservationRequired: true
status: READY
zone: https://www.googleapis.com/compute/v1/projects/OWNER_PROJECT_ID/zones/ZONE

What's next