In-memory tier overview

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

The in-memory tier is a cluster-level setting that uses specialized nodes. Before you read this page, understand instances, clusters, and nodes. We also recommend that you familiarize yourself with the limitations.

Important: To help us support your request to try out the in-memory tier feature, sign up here. This feature is available only in the Enterprise Plus edition. For more information, see Editions overview.

Bigtable in-memory is an integrated memory tier as part of the Bigtable node that enables seamless, cost-optimized data tiering based on access patterns and performance needs, all through a single API. Using remote direct memory access (RDMA), Bigtable in-memory tier significantly reduces response times, mitigates hotspots and ensures high throughput for latency-sensitive data. For more information, see Understand performance.

When to use in-memory

We recommend that you enable the in-memory tier for the following use cases:

• Sub-millisecond latency requirements: workloads that require read latencies of less than 1ms, typically addressed by fronting a database with a self-managed cache solution.
• High-throughput point reads: applications with high read traffic throughput requirements on specific subsets of data.
• Hotspot mitigation: high throughput workloads that may include traffic spikes or hotspots on particular row keys.
• Cost optimization: scenarios where you want to reduce the number of Bigtable nodes by offloading high read volumes to a more cost-effective in-memory tier.

How the in-memory tier works

In-memory support is provided by hybrid storage nodes. These specialized nodes extend Bigtable's compute and storage disaggregation to include a memory tier with vertically scalable throughput. In-memory can take up to 30 minutes to provision the specialized nodes. During this period, Bigtable continues to serve traffic from persistent storage.

You can enable the in-memory tier at the cluster level.

In-memory tier operates as follows:

• Getting data in the memory tier: in-memory offers read-through caching behavior. A row goes into the memory tier with read operations through an app profile that you configure for in-memory. A write operation from any app profile updates the memory tier if the row data is already in the in-memory tier.
• Eviction and invalidation policies: in-memory uses Least Recently Used (LRU) eviction at the row level and a 15-minute time to live (TTL) invalidation policy.
• Negative caching: in-memory remembers if a resource is missing for a period of time to prevent backend overload.
• Security: in-memory supports encryption at rest and in transit (TLS).
• Replication: in a multi-cluster instance, memory tiers of different clusters aren't in sync. Because each memory tier independently determines which data to load into memory based on the read traffic it receives, different clusters might hold different row keys in their memory tiers. Bigtable loads data into memory from the local cluster's SSD tier. A write to a row eventually updates it in all memory tiers where it is present, following the standard Bigtable replication model of eventual consistency.
• Vertical autoscaling: the in-memory tier supports vertical autoscaling at the node level that works alongside Bigtable autoscaling.

The in-memory tier is optimized for high-performance access to current data. To maintain this performance, only the latest timestamped versions of a row reside in the memory tier. Requests for earlier versions of data, also known as historical cells, automatically bypass the memory tier and are served from persistent storage.

Data coherence

Bigtable ensures that in-memory data is consistent with the persistent data on SSD or infrequent access storage tier. This hybrid architecture lets Bigtable provide a consistent storage model across all storage tiers, enabling you to access your data through the same semantic interface regardless of whether it physically resides in RAM or persistent storage. With Bigtable in-memory enabled, you retain read-your-write consistency at the cluster level.

Vertical scaling

Enabling the in-memory tier lets each node use vertical scaling to modify its in-memory throughput capacity. Each node includes 8 GB of RAM for the in-memory tier and starts with a base capacity of 40,000 reads per second. To manage surges in point-read traffic, nodes can automatically scale in increments of 40,000 reads per second, reaching up to a maximum of 120,000 reads per second per node.

This vertical scaling feature integrates with Bigtable autoscaling. If the maximum in-memory capacity is reached on all nodes in the cluster and autoscaling is active, Bigtable automatically provisions additional nodes to further boost in-memory throughput.

Vertical scaling on a node only applies to in-memory reads per second, and it works whether or not cluster-level autoscaling is enabled. Vertical scaling beyond the base 40,000 reads per second is billed by applying a multiplier to the node's hourly cost. For more information, see pricing and limits.

Limitations

The following limitations apply to the in-memory tier:

• In-memory requires SSD storage.
• In-memory supports only single row-level point read operations.
• In-memory doesn't serve data scans or support SQL queries.
• In-memory app profiles support only single-cluster routing.
• In-memory limits row size to 1 MiB per row key. Bigtable reads rows that exceed this limit from persistent storage.
• In-memory requires version 2.77.0 or later of the Bigtable client library for Java or BOM version 26.80.0 or later.
• Customer-Managed Encryption Keys (CMEK) clusters don't support in-memory.

What's next

• Learn how to enable the in-memory tier