NVIDIA NCP-AIO key terms

A purpose-built data center designed end to end for training and serving AI models, where compute, networking, storage, power, and cooling are engineered as one system rather than assembled from general-purpose parts.

A validated design document describing how components such as servers, switches, cabling, and software are combined for a specific workload, giving deployment teams a known-good baseline to replicate and adapt.

A group of compute nodes connected by a high-speed network and managed as a unit so that a single training or inference job can use all of their GPUs together.

A single server within a cluster, typically containing multiple GPUs, CPUs, memory, local storage, and one or more network adapters used for data and management traffic.

A processor with thousands of parallel cores optimized for matrix and vector math, making it the primary accelerator for training and running neural networks.

Any specialized processor, such as a GPU or dedicated AI chip, added to a server to perform parallel math faster than a general-purpose CPU can, offloading the main workload.

The general-purpose processor in a GPU server that runs the operating system, drivers, and orchestration agents, and coordinates data movement to and from the accelerators.

The repeating execution block inside an NVIDIA GPU that contains CUDA cores, tensor cores, registers, and shared memory, and schedules warps of threads.

A scalar arithmetic unit inside a streaming multiprocessor that executes floating-point and integer instructions for the threads of a warp, forming the GPU's general parallel compute capacity.

A specialized unit in modern NVIDIA GPUs that performs small matrix multiply-accumulate operations in a single instruction, greatly accelerating the dense math used by deep learning.

A group of 32 threads that a GPU schedules and executes together in lockstep; divergence within a warp reduces efficiency because paths are serialized.

A function launched by the host to run in parallel across many GPU threads; kernels are the basic unit of work in CUDA programs and dominate GPU execution time.

NVIDIA's parallel computing platform and programming model, including compilers, libraries, and runtime APIs, that lets developers write code executed on GPUs.

NVIDIA's GPU-accelerated library of primitives for deep neural networks, providing tuned implementations of convolution, pooling, normalization, and activation operations used by major frameworks.

NVIDIA's collective communication library that implements operations such as all-reduce and all-gather efficiently across GPUs, whether inside one node or across a network fabric.

A coordinated data exchange among many processes, such as broadcast, all-reduce, or all-gather, used to synchronize gradients and parameters during distributed training.

A collective operation that combines values from every participating rank, for example summing gradients, and delivers the combined result back to all ranks.

A training strategy where each GPU holds a full model copy, processes a different batch slice, and gradients are averaged across workers before each optimizer step.

A training strategy that splits a single model across multiple GPUs, either by layers (pipeline) or within layers (tensor), used when the model does not fit on one device.

Training or inference that uses lower-precision formats such as FP16 or BF16 for most math while keeping some values in FP32, improving throughput and reducing memory use.

The high-bandwidth memory attached directly to an accelerator, holding model weights, activations, gradients, and caches; capacity and bandwidth often limit model size and batch size.

A stacked DRAM technology mounted close to the GPU die on an interposer, delivering much higher bandwidth than conventional graphics memory for bandwidth-hungry AI workloads.

The rate at which data can be read from or written to memory, usually in gigabytes per second; many inference workloads are limited by it rather than by compute.

A performance analysis chart that compares an application's arithmetic intensity with hardware peak compute and memory bandwidth to show whether it is compute-bound or memory-bound.

A high-speed direct GPU-to-GPU interconnect that lets accelerators in a node or rack exchange data far faster than going through PCIe, reducing communication overhead.

A switch chip that connects many NVLink links so that all attached GPUs can communicate with each other at full bandwidth, including non-adjacent pairs.

The standard expansion bus connecting CPUs, GPUs, network cards, and storage inside a server; its generation and lane count determine available host-to-device bandwidth.

The adapter that connects a server to the network; in AI clusters, high-speed adapters such as InfiniBand or RoCE cards carry training and storage traffic.

A high-throughput, low-latency switched network standard widely used for AI clusters, offering remote direct memory access and congestion control suited to collective traffic.

A technology that carries remote direct memory access traffic over Ethernet, allowing low-latency, low-CPU data transfer on standard Ethernet fabrics when configured with lossless features.

A capability that lets a network adapter transfer data directly to or from GPU memory without copying through host memory, cutting latency and CPU load in multi-node communication.

The complete switched network of links and switches that interconnects nodes and storage in a cluster, including its topology, cabling, and routing configuration.

A cluster network design where each GPU in a node connects to a separate parallel leaf switch rail, so identical GPU indices across nodes communicate over few hops.

A two-tier network design where every leaf switch connects to every spine switch, giving predictable, uniform latency and easy scaling for east-west cluster traffic.

The ratio between total edge port bandwidth and available uplink bandwidth in a network tier; higher ratios lower cost but can bottleneck collective communication.

Software such as Slurm or a Kubernetes-based system that queues workloads, allocates nodes and GPUs, and enforces priorities and quotas across a shared cluster.

A feature that partitions one physical GPU into isolated instances with dedicated memory, cache, and compute slices, letting multiple small workloads share a device safely.

A packaged, isolated user-space environment bundling an application with its libraries; AI workloads rely on containers plus GPU runtime hooks to access accelerators reproducibly.

The kernel-level software that initializes GPUs, manages memory and contexts, and exposes devices to user-space libraries and containers.

The maximum heat a component such as a GPU is expected to dissipate under sustained load, which drives cooling capacity, airflow, and rack power planning.

Cooling approach that removes heat with circulating fluid via cold plates or immersion rather than air alone, enabling dense racks of high-power accelerators.

The ratio of total facility power to the power delivered to IT equipment; a lower value means less energy is lost to cooling, conversion, and distribution overhead.

A design in which compute trays, switches, power shelves, and cooling are integrated as a single rack-level system, simplifying cabling and enabling very high GPU density.

A deep learning library such as PyTorch or TensorFlow that provides model building, automatic differentiation, and training loops on top of GPU libraries.