Cloud Storage offers a variety of options to help you optimize your data storage for use cases that require high performance, such as artificial intelligence (AI), machine learning (ML), and data-intensive analytics. This page introduces you to these options and guides you through selecting an option that's appropriate for the needs of your workloads.
The Cloud Storage Rapid product family
Cloud Storage Rapid is a family of high-performance products in Cloud Storage that are designed to clear data bottlenecks for your AI/ML and data analytics workloads. As AI/ML models grow in complexity and datasets reach petabyte scales, storage performance often becomes the primary bottleneck for expensive GPU and TPU clusters. Cloud Storage Rapid leverages zonal architectures to bring your data closer to your compute resources, helping your accelerators stay saturated, reduce training times, and lower the total cost of ownership (TCO) for your most demanding workloads. Cloud Storage Rapid includes Rapid Bucket and Anywhere Cache.
Store data in the Rapid storage class by using Rapid Bucket
Rapid Bucket is a high-performance, zonal object storage solution that lets you store data in the Rapid storage class by defining a zone as a bucket's location. Rapid Bucket lets you colocate your data in the same physical zone as your AI accelerators and achieve the lowest latency and highest throughput among other Cloud Storage product offerings. Rapid Bucket delivers sub-millisecond latency, up to 15 TB/s of aggregate throughput, and supports up to 20 million queries per second (QPS).
Rapid Bucket supports stateful, streaming object appends, allowing applications to write data to existing objects in real-time. This capability lets you avoid expensive object rewrites and enables immediate data visibility, allowing consumer applications to read data as it's being written.
Best used for: training, checkpointing, and serving AI models that require ultra low latency and high deterministic performance
Use this option if: you want dedicated high performance for both reads and writes
To learn how to create zonal buckets with Rapid Bucket, see Create zonal buckets.
Create zonal caches with Anywhere Cache
Anywhere Cache is a fully managed, SSD-backed zonal read cache that works with your existing regional, dual-region, or multi-region buckets without requiring API changes. A cache gives you temporary storage capacity and bandwidth that automatically scale up or down according to the needs of your workloads. Data is automatically ingested into the cache from your bucket when the data is read by a VM that resides in the same zone as the cache.
Anywhere Cache provides 2.5 TB/s of aggregate throughput and reduces latency for faster reads by colocating your data in the same zones as your VMs. In addition to serving data faster, data served from a cache can incur lower data transfer fees, data retrieval fees, and operational fees than data that's served directly from a bucket. For more information, see Benefits for Anywhere Cache.
Best used for: training, checkpointing restores, and serving AI models that require high read bandwidth from an existing Cloud Storage bucket
Use this option if: you have existing data storage in a Cloud Storage bucket that you want to accelerate
To learn how to use Anywhere Cache, see Create and manage caches.