This document helps you select an appropriate storage option for your AI agents based on their specific data lifecycle needs and latency requirements.
For implementation details, see Manage Agent Sandbox storage.
Considerations for choosing a storage solution
When choosing a storage solution for your AI agents, you should consider your agentic platform requirements, such as performance and scale, and your agents' data management requirements.
Platform requirements
Evaluate the following operational and architectural requirements of your platform:
- Platform scale and agents churn frequency (control plane): the number of concurrent agents, and the number of agents created, paused, reactivated, and deleted per minute. Platforms creating thousands of agents per minute or pausing idle agents require storage with low-latency attach and mount operations at significant scale (for example, Filestore mounts faster than Hyperdisk can attach).
- Dataset size and load latency (data plane): agents loading multi-gigabyte datasets or heavy libraries (such as Node.js or Python packages) during startup require high storage I/O performance for reading the data in seconds (for example, Hyperdisk offers high per-disk read throughput).
- Agent cold-start and reactivation latency: the expected latency, such as sub-second or multi-second. Achieving sub-second startup latency requires using GKE Agent Sandbox Warm Pools. Using direct sandbox creation generally incurs a multi-second delay for Pod startup and dynamic disk attachment.
- Storage size per agent: depending on the service chosen, you must accommodate provisioning limits, such as a minimum size of 4 GiB for Google Cloud Hyperdisk or a minimum of 10 GiB for a single Filestore share.
- Data access modes and isolation: how the platform should support workspace isolation and collaborative workspaces. This determines whether your agents will need private isolated workspaces (ReadWriteOnce), collaborative workspaces (ReadWriteMany), or exploration branching workspaces (Read-Only template with a writable scratchpad).
- Resilience: if your agents specifically require regional resilience, Filestore Multishares for GKE (Enterprise) is the appropriate choice.
- Storage cost: storage services vary significantly in price, with Hyperdisk Balanced offering a cost-efficient option compared to Filestore Multishares.
Agent data lifecycle patterns
When you decide how your solution handles persistent and ephemeral data, consider the following agent data lifecycle patterns:
- Stateful workspace (continuous state): the workspace maintains a continuous state across sessions. The agent preserves its data when it is paused (the Agent Sandbox is deleted) and restores that data from the latest saved state when reactivated (the Sandbox is re-created).
- Point-in-time restore and ownership transfer (snapshot state): the workspace acts as a snapshot state, meaning it branches off from a frozen point in time. The workspace is initialized from a historical dataset or another user's shared state, to execute an ownership transfer. Subsequent modifications are saved to a separate, private writable layer, leaving the master copy untouched. This pattern is useful for scenarios like cloning a dataset to run parallel experiments, debugging, or performing independent work based on shared data.
- Ephemeral workspace (scratch state): the workspace provides a temporary scratch state where no data is preserved. The agent uses a storage volume strictly for holding temporary files while it is active. When the agent is paused or deleted (Agent Sandbox deletion), the temporary data is permanently discarded.
Agent data access modes
Choose a storage service that supports your agents' needs if they need to access storage in one of these data access modes:
- Private isolated workspace: an agent starts up with a private isolated storage directory it has sole read and write access to.
- Collaborative workspace: multiple coordinating agents mount the exact same shared directory in Read-Write (RW) mode to collaboratively update files in real time.
- Exploration branching workspace: the agent accesses base template files in
Read-Only (RO) mode to avoid altering the template, and makes new writes
either by routing them to a separate local
emptyDirscratchpad or private persistent path or by copying the template files directly into a private writable workspace on startup.
Compare storage options for Agent Sandboxes
Consider the following scenarios to help you compare your storage options:
- Use Hyperdisk Balanced for cost-efficient storage for agents that tolerate a few seconds startup latency and use a private isolated workspace with ReadWriteOnce (RWO) access mode.
- Use Filestore Multishares for GKE (Enterprise) for agents that require a collaborative workspace or regional resilience.
The following table compares storage services to help you meet the performance, scale, data access, and cost requirements of your AI agents.
| Feature | Hyperdisk Balanced | Filestore Multishares for GKE (Enterprise) |
|---|---|---|
| Best for |
|
|
| Access modes |
ReadWriteOnce (RWO) Note: Use Hyperdisk ML for ReadOnlyMany (ROX) mode. |
ReadWriteMany (RWX) |
| Sub-second Agent Sandbox startup (Warm Pools) |
|
|
| Storage provisioning latency | Several seconds per volume |
|
| Hot path attachment and mount latency | Several seconds for disk attachment | Sub-second for Network NFS mount |
| Maximum read throughput |
|
|
| IOPS | From 3,000 to 160,000, depending on volume size and configuration |
|
| Size limits |
|
|
| Scale limits |
|
|
| Capacity scaling direction | Only scales up | Scales up or down |
| CSI VolumeSnapshot support | Supported | Not supported (per-share snapshots are not supported) |
| Price | Persistent Disk and Google Cloud Hyperdisk pricing | Filestore pricing |