NVIDIA NCP-AIN flashcards
NVIDIA NCP-AIN · Build recall with a focused review session.
33 cards. Read the term and the definition here. The deck at the top of this page is optional.
- InfiniBand credit-based flow control
- InfiniBand uses link-level credits so a sender never overruns a receiver's buffer; the fabric stays lossless and congestion is handled by the subnet manager and congestion control rather than packet drops.
- RoCEv2
- RDMA over Converged Ethernet version 2 encapsulates InfiniBand transport over UDP/IP on port 4791. It is routable across Ethernet networks but requires PFC and ECN/DCQCN to remain lossless under load.
- NVIDIA SHARP
- Scalable Hierarchical Aggregation and Reduction Protocol runs collective operations such as all-reduce inside Quantum InfiniBand switches, reducing host round-trips, network traffic, and CPU overhead during distributed training.
- GPUDirect RDMA
- A capability letting a network adapter DMA data directly into GPU memory over PCIe, bypassing host CPU copies. It lowers latency and CPU load for multi-node training and needs matching drivers and pinned memory.
- Subnet manager role
- The subnet manager discovers the InfiniBand topology, assigns local identifiers, programs switch forwarding tables, and configures QoS. One SM is master while others remain standby for failover.
- Rail-optimized topology
- A cabling scheme where each GPU in a node connects through its own NIC to a dedicated rail switch, so a collective step's traffic stays on one rail and avoids cross-rail congestion.
- InfiniBand port speeds
- Per-port signaling generations: HDR is 200 Gb/s, NDR is 400 Gb/s, and XDR is 800 Gb/s. Earlier generations include EDR at 100 and FDR at 56 Gb/s.
- PUE formula
- Power Usage Effectiveness equals total facility power divided by IT equipment power. A value of 1.0 is ideal; efficient modern data centers typically report roughly 1.1 to 1.4.
- ASHRAE inlet temperature range
- ASHRAE TC 9.9 recommends inlet air temperatures of 18 to 27 degrees Celsius (about 64 to 81 Fahrenheit) for Class A1 and A2 equipment; exceeding it increases fan power and throttling risk.
- N+1 versus 2N
- N+1 provides one spare unit beyond the required capacity, while 2N fully duplicates every path or component. Dual-corded AI servers on separate A and B PDUs follow the 2N pattern.
- GPU rack density
- An eight-GPU HGX or DGX-class server commonly draws 6 to 10 kW, which pushes racks past air-cooling limits and drives adoption of direct-to-chip liquid cooling or rear-door heat exchangers.
- DGX SuperPOD architecture
- A reference design combining DGX compute nodes with Quantum InfiniBand leaf and spine switches, UFM fabric management, and a validated cabling and power plan for large-scale AI training.
- DCGM monitoring scope
- NVIDIA Data Center GPU Manager collects GPU temperature, power draw, utilization, ECC errors, and xid errors, feeding operations dashboards and alerting for cluster health.
- Cable media selection
- Direct attach copper suits short in-rack runs, active optical cables cover mid distances, and optical transceivers with fiber handle long inter-row links. Bend radius and airflow are common field issues.
- CUDA warp size
- A warp contains 32 threads that execute one instruction stream together. If threads in a warp take different branches, the paths serialize, reducing effective throughput.
- BF16 versus FP16
- BF16 keeps the FP32 exponent range with fewer mantissa bits, avoiding overflow in large-model training. FP16 offers more mantissa precision but a narrower range, so it often needs loss scaling.
- Tensor parallelism
- A parallelism strategy that splits individual layers, such as attention heads or weight matrices, across GPUs. It generates heavy all-reduce traffic and is usually confined within a node connected by NVLink.
- NCCL collectives
- NVIDIA Collective Communications Library implements all-reduce, all-gather, reduce-scatter, and broadcast, selecting ring or tree algorithms based on detected topology and message size.
- Strong versus weak scaling
- Strong scaling keeps the problem size fixed while adding GPUs, so communication overhead limits gains. Weak scaling grows the problem with the hardware, keeping per-GPU work constant.
- FlashAttention purpose
- A tiled attention algorithm that reduces memory traffic between GPU high-bandwidth memory and on-chip SRAM, improving speed and memory efficiency for long sequences whose attention cost grows with length squared.
- Inference quantization
- Reducing weight and activation precision to FP8, INT8, or INT4 lowers memory footprint and raises throughput, trading some accuracy. It is typically paired with batching and KV-cache reuse.
- CUDA memory hierarchy
- Registers and shared memory are small, on-chip, and fast; global HBM memory is large but high latency. Coalesced global accesses across a warp maximize achieved DRAM bandwidth.
- NVIDIA AI Enterprise
- A licensed, supported software suite including CUDA, Triton, TensorRT, NIM, and vGPU, certified on VMware vSphere and major clouds. It is licensed per GPU or per core.
- MIG versus vGPU
- MIG partitions an Ampere or newer GPU into isolated instances with dedicated memory slices and SMs. vGPU instead time-slices a GPU, giving each virtual GPU a fixed framebuffer share.
- Cloud RDMA requirement
- Multi-node training in the cloud needs an RDMA-capable interconnect such as Elastic Fabric Adapter or InfiniBand. Ordinary TCP virtual networking bottlenecks all-reduce communication between nodes.
- Spot versus reserved instances
- Spot instances are cheap but preemptible, so they suit checkpointed training jobs. Reserved or committed-use capacity costs more but gives stable availability and latency for inference serving.
- NGC container compatibility
- NVIDIA NGC provides containerized frameworks and tools. A container's CUDA version must be compatible with the host driver, so driver versions are validated before deploying GPU workloads.
- NIM microservices
- NVIDIA Inference Microservices package optimized models as containers with standardized APIs, handling serving, batching, and runtime optimization so applications can call inference endpoints without custom serving code.
- InfiniBand LID and GID
- A local identifier is a 16-bit address assigned by the subnet manager for routing within a subnet. A global identifier is a 128-bit IPv6-format address used for global addressing.
- NVLink versus InfiniBand
- NVLink is the high-bandwidth intra-node GPU interconnect, such as 900 GB/s bidirectional per GPU on H100. InfiniBand is the inter-node fabric connecting servers through adapters and switches.
- PFC and ECN
- Priority Flow Control pauses a specific traffic class to prevent drops, while Explicit Congestion Notification marks packets so DCQCN can reduce send rate. Both are needed for lossless RoCEv2.
- DGX burn-in validation
- Acceptance testing for an AI cluster typically runs NCCL all-reduce benchmarks and GPU stress workloads to validate fabric health, thermal behavior, and sustained performance before production handoff.
- PUE efficiency interpretation
- Because PUE divides total facility power by IT power, a lower number means less overhead energy for cooling and distribution. A PUE of 1.2 means 20 percent overhead beyond IT load.