AI Hypercomputer overview

AI Hypercomputer is an integrated supercomputing system that's optimized to support your artificial intelligence (AI) and machine learning (ML) workloads. It is the unified portfolio for Google Cloud's AI infrastructure, and provides a single entry point to explore, configure, and deploy performance-optimized hardware, open software, and flexible consumption models.

AI Hypercomputer incorporates systems-level design and best practices to boost efficiency across the entire AI development lifecycle—from prototyping and development to massive-scale training and real-time serving.

System architecture

AI Hypercomputer vertically integrates these three components to maximize goodput—the measure of actual machine learning productivity:

  • Performance-optimized infrastructure: provides accelerators (NVIDIA GPUs and Google Cloud TPUs), networking, and storage resources.
    • GPUs: categorized by how the system handles their lifecycle events and maintenance:
      • General GPUs: infrastructure managed as independent compute units with asynchronous maintenance.
      • Clustered GPUs: high-performance clusters managed as a single tightly coupled system with synchronized maintenance.
    • TPUs: use hardware designed for performing large matrix operations and have on-chip high-bandwidth memory (HBM) for larger models and batch sizes. TPUs can be connected in groups called slices that scale up your workloads with little to no code changes. For information about TPU infrastructure, see the Cloud TPUs documentation.
  • Open software: offers optimized versions of machine learning frameworks (PyTorch, JAX, and TensorFlow) and orchestration platforms. To deploy and manage your accelerators, you can choose from the following options:
    • Google Kubernetes Engine for automated container-native scaling
    • Slurm through Cluster Director
    • Cluster Toolkit blueprints
  • Consumption options: offers flexible provisioning options based on your workload needs:
    • Flex-start VMs, Spot VMs, and Reservations: provide flexible options for different workloads.
    • Dynamic Workload Scheduler: provides atomic provisioning for entire blocks of interconnected nodes for large-scale training.

Benefits

  • High performance and goodput: optimize the scheduling and runtime layers to maximize goodput, helping accelerators spend more time on productive computation and less time on system overhead.
  • Integrated reliability: gain access to specialized reliability features:
    • Clustered GPUs: includes automated hardware health monitoring and proactive node replacement.
    • General GPUs: uses standard Google Cloud node auto-repair to maintain pod-level uptime for serving fleets.
  • Optimized storage: use high-throughput storage services, such as Google Cloud Managed Lustre and Cloud Storage Rapid, to eliminate I/O bottlenecks.

Use cases

AI Hypercomputer meets the needs of these use cases:

Use case Example workloads
Large-scale AI and ML workloads
  • Distributed training for generative AI
  • Inference for generative AI
  • Fraud detection
  • Recommendation models
Inference and model serving
  • Fraud detection in real time
  • Recommendation engines for results in real time
Prototyping and development
  • Small-scale experiments
  • Early-stage development

What's next