NVIDIA NCP-AIO flashcards

33 cards. Read the term and the definition here. The deck at the top of this page is optional.

PUE definition

Power usage effectiveness equals total facility power divided by IT equipment power. A value of 1.0 is theoretical perfection; AI halls with liquid cooling typically achieve lower PUE than air-cooled halls.

Rack power density limit

Air cooling becomes impractical above roughly 30-40 kW per rack. Dense 8-GPU AI nodes at 6-10 kW each push racks past 100 kW, requiring direct liquid cooling.

N+1 versus 2N

N+1 provides one spare component beyond the minimum needed; 2N fully duplicates every component. AI clusters often use N+1 power distribution and 2N utility feeds.

CDU function

A coolant distribution unit isolates facility water from the technology cooling loop serving cold plates, regulating flow, pressure, and temperature while enabling leak detection.

GPU inlet temperature target

ASHRAE A1/A2 envelopes place GPU inlet air around 18-27 C. Exceeding the upper bound causes clock throttling and reduced sustained training throughput.

Stranded capacity

Capacity that cannot be used because one resource (power, cooling, or space) is exhausted before the others, leaving installed equipment underutilized.

Cluster bring-up order

Commission power and cooling, configure BMC addressing, apply firmware baseline, install OS and GPU driver, start fabric manager, then deploy the container runtime and scheduler.

Golden firmware BOM

A recorded, validated set of GPU, NIC, and switch firmware versions applied cluster-wide. Mismatched firmware causes intermittent link flaps and job failures.

Node acceptance burn-in

Sustained compute and memory stress plus a multi-GPU NCCL all-reduce run, performed before a node enters production, to expose thermal, power, or fabric defects.

Gang scheduling

A scheduler policy that starts all ranks of a multi-node job simultaneously. Without it, partially allocated jobs hold GPUs while waiting for remaining resources.

XID error

A GPU error code reported in the system log by the driver. An XID entry identifies a specific hardware or driver fault on that GPU and guides triage.

Usable GPU-hours formula

Nodes times GPUs per node times hours times utilization factor. Plan 10-20 percent headroom for failures, maintenance, and scheduling fragmentation.

Occupancy definition

Active warps per streaming multiprocessor divided by the maximum supported warps. Low occupancy from register or shared-memory pressure limits latency hiding.

Roofline model

Attainable performance equals the minimum of peak FLOPs and arithmetic intensity times peak memory bandwidth. Kernels below the ridge point are memory-bound.

Arithmetic intensity

Floating-point operations performed divided by bytes moved from memory. It determines whether a kernel is limited by compute throughput or by memory bandwidth.

Tensor core requirement

Tensor cores accelerate matrix multiply when inputs use supported reduced-precision types such as FP16, BF16, or FP8 with FP32 accumulation, and tile dimensions align.

MIG partitioning

Multi-Instance GPU splits one GPU into isolated slices with dedicated memory and SMs. It suits multi-tenant inference but blocks cross-slice NVLink communication needed for training.

ECC correctable versus uncorrectable

Correctable HBM errors are logged and row-remapped without disruption; uncorrectable errors typically require GPU reset or replacement and force node drain.

Nsight Systems versus Compute

Nsight Systems shows a timeline of kernels, gaps, and synchronization stalls. Nsight Compute reports per-kernel metrics like achieved occupancy and memory throughput.

Multi-GPU scaling efficiency

Speedup measured on N GPUs divided by N. Values below one indicate communication overhead, load imbalance, or synchronization cost in the parallel workload.

InfiniBand versus RoCEv2

InfiniBand provides native lossless link-level flow control. RoCEv2 runs over Ethernet and needs priority flow control plus ECN marking to behave losslessly.

Fat-tree oversubscription

Downstream bandwidth divided by upstream bandwidth. AI training fabrics are typically built 1:1 non-blocking so all-reduce collectives are not throttled at uplinks.

Rail-optimized topology

GPU index N on every node connects to the same leaf switch rail, so all-reduce traffic between matching ranks traverses one hop instead of crossing the spine.

GPUDirect RDMA

Network adapters DMA directly into GPU memory, bypassing host copies. It requires IOMMU passthrough, peer-to-peer support, and NIC placement in the same PCIe domain.

NCCL transport selection

NCCL automatically chooses NVLink, InfiniBand, RoCE, PCIe, or shared memory. Environment variables such as NCCL_IB_HCA and NCCL_DEBUG control and diagnose the choice.

All-reduce ring cost

A ring all-reduce moves approximately 2(N-1)/N times the message size per rank. The slowest link sets collective time, so one degraded port throttles everyone.

Lossless Ethernet prerequisites

Enable priority flow control on a dedicated priority, configure ECN marking, disable energy-efficient Ethernet, and keep MTU consistent, typically 9000 bytes, end to end.

Fabric manager role

A service that configures NVLink partitions and registers the NVSwitch fabric. Without it running, multi-GPU NVLink communication on switch-based systems fails.

CUDA driver compatibility

The installed driver must meet the minimum version required by the CUDA toolkit. Minor-version compatibility allows newer toolkits on older drivers within limits.

Container toolkit injection

The NVIDIA Container Toolkit exposes GPUs and driver libraries to containers. The host driver must be compatible with the CUDA version baked into the image.

Parallelism strategy split

Data parallel replicates the model across ranks; tensor parallel splits layers within a node over NVLink; pipeline parallel splits layers across nodes. Large models combine all three.

BF16 versus FP16

BF16 keeps the FP32 exponent range with a smaller mantissa, avoiding FP16 loss-scaling problems. FP8 on newer GPUs needs per-tensor or per-block scaling.

Sharded checkpointing

Checkpoints are written in parallel shards rather than by a single writer, removing the write bottleneck and allowing the model to be resharded on restart.