NVIDIA NCP-AII key terms

A path that lets GPUs read and write storage directly, bypassing the CPU and host memory bounce buffer. It lowers latency and CPU load for data-heavy AI pipelines such as checkpoint loading and dataset streaming.

A capability that lets a network adapter transfer data straight into or out of GPU memory over PCIe, skipping host memory copies. It reduces latency and CPU overhead in multi-node training communication.

NVIDIA's collective communication library used by distributed training frameworks. It implements all-reduce, all-gather, broadcast, and similar operations, choosing rings, trees, and transport paths automatically based on detected topology.

A high-bandwidth GPU-to-GPU interconnect that lets accelerators share memory traffic without going through PCIe. On multi-GPU servers it forms the fast fabric used for model-parallel and data-parallel collectives.

A switch chip that connects many GPUs in one server at full NVLink bandwidth, creating an all-to-all fabric. It allows any GPU pair to communicate without PCIe bottlenecks or topology-dependent detours.

A low-latency, high-throughput switched network standard widely used for AI training clusters. It offers remote direct memory access, credit-based flow control, and fabric management through a subnet manager.

RDMA over Converged Ethernet version 2, which carries RDMA traffic over routable UDP/IP Ethernet networks. It requires careful configuration of priority flow control and congestion control to avoid packet loss.

An Ethernet link-level mechanism that pauses a specific traffic class instead of the whole port. It prevents RDMA packet drops on lossless RoCE fabrics but can spread congestion if misconfigured.

An IP-level signal in which switches mark packets rather than dropping them when queues build. Endpoints then slow down, which is the preferred congestion control method on RoCE networks.

The InfiniBand fabric service that discovers nodes, computes routes, and programs switch forwarding tables. Without an active subnet manager, an InfiniBand fabric cannot pass traffic between ports.

A hierarchical switch layout in which every host reaches every other host through the same number of hops at full bisection bandwidth. It is the standard design for large GPU training clusters.

A cabling scheme where each server's Nth GPU port connects to the same leaf switch group as every other server's Nth GPU. It keeps collective traffic local to a rail and reduces cross-switch hops.

The total network capacity available between two equal halves of a cluster. It sets the ceiling for all-reduce throughput when many nodes exchange gradients at the same time.

A collective operation in which every participant ends with the same reduced result, such as a summed gradient. It is the dominant communication pattern in data-parallel training and drives network sizing decisions.

An algorithm that arranges workers in a logical ring and passes partial sums in two phases, reduce-scatter then all-gather. Its cost scales with message size and is largely independent of worker count.

The fifth PCI Express generation, offering roughly 32 gigatransfers per second per lane. It provides the host bandwidth needed for current data-center GPUs and high-speed network adapters.

Splitting a physical PCIe slot's lanes into multiple independent links, for example x16 into two x8 links. It lets a server host more GPUs or NICs but can reduce per-device bandwidth.

A group of CPU cores with locally attached memory and PCIe devices. Placing a process and its GPU on the same NUMA node avoids slower cross-socket memory access during data transfers.

Pinning a process or thread to specific CPU cores so it runs consistently on chosen hardware. In GPU workloads it keeps data-loader threads near the memory and PCIe devices they use.

The ability of software to detect how GPUs, CPUs, and network adapters are physically connected and schedule communication accordingly. It avoids routing traffic across slower links when faster paths exist.

A mode that partitions one physical GPU into isolated instances, each with dedicated memory and compute slices. It lets multiple users or jobs share an accelerator with predictable quality of service.

A GPU sharing method in which multiple workloads take turns using the whole device. It improves utilization for light jobs but gives no memory isolation and can cause unpredictable latency.

A Kubernetes operator that deploys and manages GPU software components such as drivers, the container toolkit, device plugin, and monitoring agents. It automates lifecycle work that would otherwise be manual per node.

Software that exposes host GPUs to containers by integrating with container runtimes. It injects the driver libraries and device nodes a container needs without installing a full driver inside the image.

The rules governing which driver versions can run which CUDA toolkit builds. Newer toolkits generally need a minimum driver, while forward compatibility packages allow newer CUDA on older data-center drivers.

A release line of the NVIDIA data-center driver, such as a long-lived production branch or a newer feature branch. Mixing branches across nodes in one cluster can cause inconsistent behavior.

A service required on systems using NVSwitch that configures the GPU fabric and keeps it consistent. It must be started after the driver loads, and version mismatches with the driver cause startup failures.

A command-line utility that reports GPU inventory, utilization, memory use, temperature, and errors, and can set application clocks. It is the first diagnostic tool for most GPU node issues.

The Data Center GPU Manager, a toolkit that collects GPU health and performance metrics and can run diagnostic tests. It feeds cluster monitoring systems and detects issues such as thermal or memory faults.

A numeric code reported by the NVIDIA driver when a GPU encounters a hardware or software fault. Values such as 48, 63, or 79 point to specific failure classes that guide repair or reset decisions.

Memory protection that detects and corrects certain bit errors in GPU memory. Monitoring corrected and uncorrected counts helps distinguish a degrading module from a transient fault.

A sustained stress run that exercises GPUs, memory, and network links before a cluster enters production. It exposes marginal hardware and cabling faults that appear only under full load.

A structured validation performed after installation to confirm the cluster meets design requirements. It usually covers hardware inventory, link health, collective bandwidth, and storage throughput.

An open-source workload manager that allocates nodes and GPUs to jobs and enforces queue policies. It uses GRES or device configuration to grant exclusive or shared accelerator access.

An extension that advertises GPU resources to the Kubernetes scheduler and assigns them to pods. It works with the container toolkit so containers receive only the devices they requested.

A saved snapshot of model weights and optimizer state written periodically so training can resume after failure. Fast parallel checkpointing to shared storage reduces lost work on large jobs.

A storage system that spreads files across many servers to deliver aggregate throughput beyond one node. AI clusters rely on it to feed training data and absorb checkpoint bursts.

A network file system protocol for sharing directories across hosts. It is simple and common for home directories but generally lacks the parallel throughput and scaling needed for large training datasets.

Remote Direct Memory Access lets one host read or write another host's memory without involving its CPU. It delivers the low latency and high throughput that distributed training communication depends on.

The practice of separating users, namespaces, and workloads so one tenant cannot access another's data or GPUs. It combines identity controls, network policy, and device-level partitioning.

Hardware-based protection that encrypts data in use so the host operating system and administrators cannot read it. On GPUs it enables trusted execution for sensitive models and datasets.