Tune gcloud CLI performance for zonal buckets

When the Google Cloud CLI reads objects from zonal buckets in Rapid Bucket, it automatically applies default concurrency and slicing configurations based on your available system resources.

This page describes these default configurations and explains how you can use the gcloud CLI to further tune concurrency, process scaling, and opt-in features (such as NIC isolation) to help improve performance for specific workloads. For tuning performance when mounting zonal buckets as a file system, see Cloud Storage FUSE performance for Rapid Bucket.

Before configuring performance settings, make sure that you create a zonal bucket. If you plan to download objects to an attached storage volume (such as Hyperdisk or Local SSD), make sure that you format and mount the volume in your environment. For general slicing concepts, see Sliced object downloads.

Default concurrency and slicing configurations

The default configurations in this section apply specifically when reading objects from zonal buckets in Rapid Bucket using gcloud CLI version 583.0.0 or later. For standard Cloud Storage buckets, the gcloud CLI applies its own default concurrency settings.

For zonal buckets, the gcloud CLI automatically sets default concurrency and slicing parameters based on the number of available virtual CPU (vCPU) cores and whether you are downloading a single object or multiple objects. The following tables map default values for these properties:

Process count: storage/process_count
Thread count: storage/thread_count
Slicing threshold: storage/sliced_object_download_threshold
Max components: storage/sliced_object_download_max_components
Component size: storage/sliced_object_download_component_size

Single-object downloads

When downloading a single object, the gcloud CLI applies the following defaults:

Available vCPUs	Process count	Thread count	Slicing threshold	Max components	Component size
`≥ 8`	`8`	`2`	`50 MiB`	`16`	`5 MiB`
`< 8`	`4`	`2`	`50 MiB`	`8`	`5 MiB`

Multi-object downloads

When downloading multiple objects simultaneously, the gcloud CLI applies the following defaults:

Available vCPUs	Process count	Thread count	Slicing threshold	Max components	Component size
`≥ 48`	`min(96, available_cores * 0.75)`	`1`	`10 MiB`	`5`	`5 MiB`
`4 to 47`	`16`	`4`	`10 MiB`	`10`	`5 MiB`
`< 4`	`2`	`10`	`50 MiB`	`10`	`5 MiB`

Tune concurrency and slicing configurations

This section describes how to configure concurrency and slicing settings using the gcloud CLI and provides recommended values for common hardware environments.

How to apply a configuration

To configure and apply custom tuning settings, complete the following steps:

Install or update the Google Cloud CLI to version 583.0.0 or later.
In your development environment, run the gcloud config configurations create command to create and activate a configuration profile:
```
gcloud config configurations create CONFIGURATION_NAME
```
Replace CONFIGURATION_NAME with a name for your configuration profile, such as rapid-perf.
Run the gcloud config set command to configure concurrency and slicing properties:
```
gcloud config set storage/thread_count THREAD_COUNT
gcloud config set storage/process_count PROCESS_COUNT
gcloud config set storage/sliced_object_download_threshold THRESHOLD_SIZE
gcloud config set storage/sliced_object_download_component_size COMPONENT_SIZE
gcloud config set storage/sliced_object_download_max_components MAX_COMPONENTS
```
Replace the placeholders with values suited for your workload:
- THREAD_COUNT: the number of threads per worker process, such as 1.
- PROCESS_COUNT: the number of worker processes, such as 64.
- THRESHOLD_SIZE: the minimum object size threshold to trigger slicing, such as 128 MiB for multi-gigabyte workloads (or 32 MiB for smaller objects).
- COMPONENT_SIZE: the target size of each download slice, such as 128 MiB for multi-gigabyte workloads (or 32 MiB for smaller objects).
- MAX_COMPONENTS: the maximum number of slice components per object, such as 16.
For more information about these properties, see Tuning recommendations.
Download objects to your local storage path by running the gcloud storage cp command:
```
gcloud storage cp -r gs://BUCKET_NAME/SOURCE_PATH/ /DESTINATION_PATH/
```
Replace the following:
- BUCKET_NAME: the name of your zonal bucket.
- SOURCE_PATH: the source directory or object path in your bucket.
- DESTINATION_PATH: your local directory path, such as ./data/, or the mount point of your local storage volume, such as /mnt/hyperdisk/data/.

Tuning recommendations

To choose appropriate values for the placeholders in the previous steps, review the following guidelines:

Process count

Set the storage/process_count property to scale parallel worker processes based on available CPU cores. Limit the process count to a maximum of 80% of available CPU cores:

storage/process_count = min(target_cores, 0.8 * available_cores)

Where:

target_cores: the number of CPU cores or worker processes you want to allocate for your transfer (such as 64).
available_cores: the total number of virtual CPUs (vCPUs) available on your machine (for example, by running nproc on Linux).

For example, if your target is 64 worker processes, machines with 80 or more vCPUs can set storage/process_count to 64. For machines with fewer than 80 vCPUs, set storage/process_count to 80% of available cores (for example, 51 on a 64-vCPU VM).

Thread count

Set the storage/thread_count property to control the number of threads per worker process. On machines with 48 or more vCPUs, set storage/thread_count to 1.

Restricting each worker process to a single thread on high-core machines helps reduce Python Global Interpreter Lock (GIL) and gRPC thread contention.

Slicing threshold

Set the storage/sliced_object_download_threshold property to specify the minimum object size required to trigger sliced downloads.

We recommend setting storage/sliced_object_download_threshold to a value equal to or greater than storage/sliced_object_download_component_size.

Slices per object

Set the storage/sliced_object_download_component_size and storage/sliced_object_download_max_components properties to control the number of parallel slices generated per object. The gcloud CLI calculates slices per object using the following formula:

Slices per object = min(object_size / component_size, max_components)

If an object is large enough that slicing it by the component_size value would exceed the max_components value, the gcloud CLI ignores the component_size value and divides the object evenly into max_components slices. For example, downloading a 100 GiB object with component_size=128 MiB and max_components=16 produces 16 slices of 6.25 GiB each.

Each slice is downloaded independently and in parallel based on your total worker capacity (storage/process_count × storage/thread_count).

Worker saturation

To help maintain high worker utilization throughout a transfer, adjust your process count, thread count, component size, and max components so that your typical transfers generate enough total slices to equal or exceed your total worker capacity:

Total slices across all objects >= storage/process_count * storage/thread_count

For example, consider downloading four objects using 64 worker processes (process_count=64 and thread_count=1):

Four 2 GiB objects: setting component_size=128 MiB generates 16 slices per object (4 × 16 = 64 slices), fully utilizing all 64 worker processes.
Four 512 MiB objects: setting component_size=32 MiB generates 16 slices per object (4 × 16 = 64 slices). In contrast, using component_size=128 MiB on 512 MiB objects produces only 4 slices per object (16 total slices), leaving 48 worker processes idle.

NIC isolation

On Linux machines with more than 16 CPU cores, you can enable network interface card (NIC) isolation by setting the storage/use_nic_isolation property to True:

gcloud config set storage/use_nic_isolation True

When you enable this property, the gcloud CLI sets CPU affinity (os.sched_setaffinity()). This configuration isolates 10% of CPU cores for network hardware interrupt requests (IRQs) and reserves the remaining cores for data processing.