Accelerate database performance using disk cache

Select a documentation version:

This page describes how to provision, set up, and use disk caching on AlloyDB Omni to increase performance of your AlloyDB Omni installation.

In addition to the standard PostgreSQL in-memory shared buffers, AlloyDB Omni disk cache enables storing buffers on fast storage such as solid-state drives (SSDs). Disk caching accelerates data retrieval in AlloyDB Omni installations with data directories located on slower storage.

Like PostgreSQL shared buffers, AlloyDB Omni disk cache is non-persistent, which means cached data is lost on restart.

By default, AlloyDB Omni disk cache uses all storage reported by the file system. You can define the amount of storage reserved for caching data using the omni_disk_cache_file_size parameter.

Enable AlloyDB Omni disk cache

The steps you use to enable disk cache for AlloyDB Omni depend on whether you run AlloyDB Omni in a container or on a Kubernetes cluster.

Enable disk cache on a generic volume

You can enable disk cache using a generic volume.

For enabling disk cache on a generic volume on the AlloyDB Omni Kubernetes operator, you need a persistent volume ready ahead of time and a storageClass.

For example, if you use GKE and don't have the persistent volume and storageClass ready, make sure you have performed the following before enabling disk cache on a generic volume:

  1. Created a cluster with local SSD-based storage.
  2. Formatted your volume to the ext4 file system using step 1 of Run the local volume static provisioner.
  3. Manually created a persistent volume for each SSD in your cluster with the storageClass that defines persistent storage on a storage device.

To enable disk cache on a generic volume for your database, complete the following steps:

  1. Modify your database cluster manifest to add the ultraFastCache attribute to the features section of the spec section:

          apiVersion: alloydbomni.dbadmin.goog/v1
      kind: DBCluster
      metadata:
        name: CLUSTER_NAME
      spec:
        databaseVersion: "17.5.0"
        primarySpec:
          features:
            ultraFastCache:
              cacheSize: DISK_CACHE_SIZE
              genericVolume:
                storageClass: "STORAGE_CLASS_NAME"
      ...

    Replace the following:

    • DB_CLUSTER_NAME: the name of your database cluster. It is the same database cluster name you declared when you created it.
    • DISK_CACHE_SIZE: the size of the cache—for example, 100Gi. It must be greater than shared_buffers. This field is optional. If you don't specify the value of this field, AlloyDB Omni uses all space left on the disk, which applies to both AlloyDB Omni in a container and on a Kubernetes cluster.
    • STORAGE_CLASS_NAME: the name of the storage class of the ultra fast cache volume—for example, local-storage.
  2. Reapply the manifest.

Enable disk cache on a local volume

If you want to use a local volume, you don't have to create a persistent volume. You can use the following optimization instead.

For example, if you use GKE and don't have the persistent volume and storageClass ready, make sure you have performed the following before enabling disk cache on a local volume:

  1. Created a cluster with local SSD-based storage.
  2. Formatted your volume to the ext4 file system using step 1 of Run the local volume static provisioner.

To enable disk cache on a local volume for your database, do the following:

  1. Modify your database cluster manifest to add the ultraFastCache attribute to the features section of the spec section:

        apiVersion: alloydbomni.dbadmin.goog/v1
    kind: DBCluster
    metadata:
      name: CLUSTER_NAME
    spec:
      databaseVersion: "17.5.0"
      primarySpec:
        features:
          ultraFastCache:
            cacheSize: DISK_CACHE_SIZE
            localVolume:
              path: "LOCAL_VOLUME_PATH"
              nodeAffinity:
                required:
                  nodeSelectorTerms:
                  - matchExpressions:
                    - key: "LABEL_KEY"
                      operator: "OPERATOR_VALUE"
                      values:
                      - "LABEL_KEY_VALUE"
    ...

    Replace the following:

    • CLUSTER_NAME: the name of your database cluster. It is the same database cluster name you declared when you created it.
    • DISK_CACHE_SIZE: the size of the cache—for example, 100Gi. It must be greater than shared_buffers. This field is optional. If you don't specify the value of this field, AlloyDB Omni uses all space left on the disk, which applies to both AlloyDB Omni in a container and on a Kubernetes cluster.
    • STORAGE_CLASS_NAME: the name of the storage class.
    • LOCAL_VOLUME_PATH: the path to the local volume—for example, /mnt/disks/raid/0.
    • LABEL_KEY: the node's label for the key that serves as a location indicator and facilitates even Pod distribution across the cluster—for example, cloud.google.com/gke-local-nvme-ssd.
    • OPERATOR_VALUE: the key's relationship to a set of values—for example, In. Set the parameter to one of the following:
      • In: the values array must be non-empty.
      • NotIn: the values array must be non-empty.
      • Exists: the values array must be empty.
      • DoesNotExist: the values array must be empty.
      • Gt: the values array must have a single element, which is interpreted as an integer.
      • Lt: the values array must have a single element, which is interpreted as an integer.
    • LABEL_KEY_VALUE: the value for your label key—for example, true. Set the parameter to an array of string values as follows:
      • If the operator is In or NotIn, the values array must be non-empty.
      • If the operator is Exists or DoesNotExist, the values array must be empty.
      • If the operator is Gt or Lt, the values array must have a single element, which is interpreted as an integer.
  2. Reapply the manifest.

Enable disk cache on an ephemeral volume

You can enable disk cache using an emptyDir volume, which provides temporary storage that is tied directly to your pod's lifecycle. The volume is created when a pod is assigned to a node, and exists as long as the pod runs on that node.

When a pod is removed from a node, the data in the emptyDir is deleted permanently. This type of volume is suitable for a cache because it provides temporary, fast storage. The data in the cache doesn't need to be persisted if the pod is rescheduled.

To enable disk cache on an ephemeral volume for your database, complete the following steps:

  1. Modify your database cluster manifest to add the ultraFastCache attribute to the features section of the spec section, and specify emptyDirVolume:

          apiVersion: alloydbomni.dbadmin.goog/v1
      kind: DBCluster
      metadata:
        name: CLUSTER_NAME
      spec:
        databaseVersion: "17.5.0"
        primarySpec:
          features:
            ultraFastCache:
              cacheSize: DISK_CACHE_SIZE
              emptyDirVolume: {}
        ...

    Replace the following:

    • CLUSTER_NAME: the name of your database cluster. It is the same database cluster name you declared when you created it.
    • DISK_CACHE_SIZE: The size of the cache—for example, 200Gi.
  2. Reapply the manifest.

Verify the disk cache configuration

After enabling AlloyDB Omni disk cache, verify that the disk cache is accessed by monitoring read and write activity to the disks using available utilities like iotop or iostat.

Additionally, you can check if the AlloyDB Omni disk cache is open.

To verify the disk cache configuration for AlloyDB Omni, use the following command.

  kubectl exec -i $DATABASE_POD -c database -n $DBCLUSTER_NAMESPACE -- cat /obs/diagnostic/postgresql.log | grep "opened omni disk cache"

If your disk caching is configured correctly, the Successfully opened omni disk cache ... message displays in the logs.

What's next