Modern machine learning frameworks often use the
[NVIDIA Collective Communications Library (NCCL)](https://developer.nvidia.com/nccl)
for GPU-to-GPU communication primitives.

Google's enhanced version of NCCL is called NCCL/gIB and is available on
Google Cloud's A5X, A4X Max, A4X, A4, and A3 Ultra instances. NCCL/gIB is often
[more performant](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/overview.md.txt#benefits) than upstream NCCL on Google infrastructure.
Because NCCL performance can affect overall workload performance, we recommend
that you use NCCL/gIB.

NCCL/gIB contains Google-specific features and optimizations such as the following:

- The **gIB network plugin** offers improved load balancing on Google's networks, leading to high throughput and low latency more consistently during collective operations.
- A **custom tuner plugin**, which selects the best tuning options on Google Cloud VMs.
- The [**CoMMA profiler plugin**](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/comma.md.txt) provides detailed performance metrics and diagnostic data for your workload.
- The **env plugin** provides platform-specific variable configurations across Google Cloud VMs.

## NCCL/gIB architecture

NCCL/gIB interacts with your machine learning framework and the NVIDIA GPUs on your clusters to optimize performance and gather telemetry, as shown in this diagram:

![The ML workload is managed by a ML framework that connects to both the NVIDIA GPUs and NCCL, while NCCL connects to various Google tools and plugins.](https://docs.cloud.google.com/static/ai-hypercomputer/images/nccl-gib-architecture-v2.png)

## Benefits of using NCCL/gIB

Although it's possible to use the upstream NVIDIA Collective Communications Library on Google Cloud VMs without stability problems, NCCL/gIB is better optimized for Google Cloud and the performance disparity can be very significant for certain communication patterns, even with the same NCCL parameters.

For example, the following graph shows a comparison of NCCL/gIB with upstream NCCL on AllReduce performance. NCCL/gIB outperforms upstream NCCL by as much as 12x on certain message sizes.

![A graph showing NCCL/gIB outperforms upstream NCCL at AllReduce tasks.](https://docs.cloud.google.com/static/ai-hypercomputer/images/nccl-gib-all-reduce.png)

32-node NCCL AllReduce performance using A3 Ultra (H200) with no background traffic.

Similarly in a comparison of NCCL/gIB with upstream NCCL on AllGather performance with background traffic, NCCL/gIB outperforms upstream NCCL by approximately 50% on larger message sizes, as shown in this graph.

![A graph showing NCCL/gIB outperforms upstream NCCL at AllGather tasks.](https://docs.cloud.google.com/static/ai-hypercomputer/images/nccl-gib-all-gather.png)

32-node NCCL AllGather performance using A3 Ultra (H200) on a shared fabric with a noisy background.

In addition, the [**CoMMA profiler plugin**](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/comma.md.txt) provides Google with improved custom telemetry, enabling us to better assist you should a workload-level issue arise.

## Install NCCL/gIB

Before you run NCCL tests, you must
[install NCCL/gIB](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/install-nccl.md.txt) in your runtime
environment.

## Use NCCL/gIB

To run NCCL/gIB tests on your cluster, choose the page from the following list
that best meets your requirements:

- [Use gIB NCCL plugins in NVIDIA NGC containers](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/ngc-containers.md.txt)
- [Run NCCL tests on Compute Engine instances](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/test-vms.md.txt)
- [Run NCCL on GKE clusters that use Cluster Toolkit](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/test-gke.md.txt)
- [Run NCCL on GKE clusters that use A4X Max](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/test-gke-custom-a4x-max.md.txt)
- [Run NCCL on GKE clusters that use A4X](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/test-gke-custom-a4x.md.txt)
- [Run NCCL on GKE clusters that use A4 or A3 Ultra](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/test-gke-custom.md.txt)
- [Run NCCL tests on Slurm clusters](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/test-slurm.md.txt)
- [Run NCCL tests in Cluster Director](https://docs.cloud.google.com/cluster-director/docs/monitor/cluster-health.md.txt#verify-health)

For directions on how to run NCCL/gIB on GKE clusters that use A5X, contact
your Google Cloud representative.

To learn how to address any issues with your cluster after you've run your
tests, see
[Collect and understand NCCL/gIB logs for troubleshooting](https://docs.cloud.google.com/ai-hypercomputer/docs/nccl/collect-and-understand.md.txt).