To efficiently run artificial intelligence (AI) or machine learning (ML)
workloads, you must select both a workload orchestrator and a deployment
platform. These components function as follows:

- An orchestrator schedules and runs your jobs.

- A deployment option manages an orchestrator.

After you identify which GPU infrastructure (
clustered GPUs or
general GPUs) to use to run your workloads, follow
the guidance in this document to identify the orchestrator and deployment that
best fits your workload and level of management.

To review the GPU machine types that you can use to run your workloads, see [GPU
machines](https://docs.cloud.google.com/ai-hypercomputer/docs/gpu).

## Compare orchestrators and deployment options

AI Hypercomputer offers several orchestrators and deployment options
for your compute instances. These options range from fully managed clusters that
let you focus on your workloads, to self-managed environments that offer
granular control over how you maintain and update your compute instances.

The following table compares the orchestrators and deployment options that you
can use when running AI workloads:

| Product | Orchestrator | Technical complexity | Recommended for | Key benefit |
|---|---|---|---|---|
| Cluster Director | Slurm, custom, none (MIGs) | Low | AI researchers, data scientists, ML engineers | Managed lifecycle for Slurm and managed compute clusters |
| Google Kubernetes Engine (GKE) | Kubernetes | Medium to high | ML engineers, Platform engineers | Container orchestration and scaling |
| Cluster Toolkit | Slurm, Kubernetes | Medium | ML DevOps, Platform engineers | Validated infrastructure-as-code blueprints |
| Compute Engine | Custom, none | High | Infrastructure architects | Granular control over OS and networking |

## Choose an orchestrator and deployment option

To choose an orchestrator and deployment option, use the following flowchart:

[![Decision tree for selecting an orchestrator and deployment option.](https://docs.cloud.google.com/static/ai-hypercomputer/images/choose-orchestrator.png)](https://docs.cloud.google.com/static/ai-hypercomputer/images/choose-orchestrator.png)

The preceding flowchart asks the following questions:

1. Do you want cluster orchestration for your workloads?

   - **Yes**: go to question 2.
   - **No**: use Compute Engine or managed compute clusters.
2. Do you want to run standard containerized apps?

   - **Yes**: use GKE.
   - **No**: go to question 3.
3. Do you want Google to manage the cluster lifecycle?

   - **Yes**: use Cluster Director.
   - **No**: use Cluster Toolkit.

## Supported orchestrators

An orchestrator automates cluster management, and eliminates the need
to manage each compute instance individually. Orchestrators like
Slurm or Kubernetes handle job queueing, resource allocation, and
autoscaling.

- **Slurm**: an open source orchestrator commonly used for AI, ML,
  and HPC workloads. You can use Slurm with
  Cluster Toolkit or Cluster Director.

- **Kubernetes**: an open source container orchestration platform. You
  can deploy Kubernetes by using GKE directly or
  through Cluster Toolkit.

- **Custom or none**: if you prefer to use a different orchestrator,
  or if you want to manage your compute instances without an orchestrator,
  then use Compute Engine or managed compute clusters in
  Cluster Director:

  - For the highest level of control, use Compute Engine. This option requires you to set up, maintain, and update your compute instances.
  - To automate Virtual Private Cloud (VPC) and RDMA network setup, shared storage mounts, and MIG provisioning, create a managed compute cluster in Cluster Director.

## Supported deployment platforms

After you identify the recommended orchestrator and deployment option for your
use case, review their descriptions below to verify that their management levels
and features meet your workload requirements.

### Managed and automated platforms

If you want to avoid the overhead of managing a cluster and instead focus on
running your workloads, then use one of the following managed platforms:

- **Cluster Director** : a fully managed service that automates the
  lifecycle of both Slurm clusters and managed compute clusters backed by
  Compute Engine MIGs. Use Cluster Director to simplify the
  setup and configuration of compute, networking, and storage resources so
  that you can focus on running your AI, ML, or HPC workloads. For more
  information, see
  [Cluster Director overview](https://docs.cloud.google.com/cluster-director/docs/overview.md.txt).

- **GKE** : a managed Kubernetes environment optimized for AI
  and ML workloads. Use GKE to orchestrate containerized
  workloads, integrate with specialized Compute Engine hardware, and
  leverage GKE-specific features like Topology Aware Scheduling (TAS). For
  more information, see [GKE overview](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/kubernetes-engine-overview.md.txt).

### Semi-managed and self-managed platforms

If you require granular control over the operating system, kernel
configurations, or driver versions, then use a semi-managed or self-managed
platform:

- **Cluster Toolkit** : an open source toolkit that provides
  validated, customizable blueprints for deploying reproducible AI and ML
  environments. It offers high flexibility and control using
  infrastructure-as-code (IaC) principles. For more information, see [Cluster Toolkit overview](https://docs.cloud.google.com/cluster-toolkit/docs/overview.md.txt).

- **Compute Engine** : a customizable compute service that provides maximum
  control for building custom environments from the ground up. While not a
  standalone orchestrator, Compute Engine serves as the foundation for users
  who prefer to build and manage their own specialized custom stacks. For more
  information, see [Compute Engine overview](https://docs.cloud.google.com/compute/docs/overview.md.txt).

## What's next

- To obtain capacity, see [Consumption options](https://docs.cloud.google.com/ai-hypercomputer/docs/consumption-models.md.txt).
- To deploy your cluster, see [Cluster creation overview](https://docs.cloud.google.com/ai-hypercomputer/docs/process-overview.md.txt).