NVIDIA NCA-AIIO flashcards

33 cards. Read the term and the definition here. The deck at the top of this page is optional.

Machine learning

A branch of AI in which algorithms learn patterns from data and improve performance on a task, instead of being programmed with explicit rules for every case.

Deep learning

A machine-learning approach using neural networks with many stacked layers that learn features automatically from raw data such as images, audio, or text.

Supervised learning

A training paradigm that learns a mapping from labeled input-output pairs by minimizing the difference between predicted values and known targets such as class labels or numbers.

Reinforcement learning

A training paradigm in which an agent takes actions in an environment and learns a policy that maximizes cumulative reward through trial, error, and feedback.

F1 score

The harmonic mean of precision and recall, computed as F1 = 2PR/(P+R); it balances false positives against false negatives in a single classification metric.

Epoch

One complete pass of the entire training dataset through the neural network during training; models are usually trained for many epochs until convergence.

Learning rate

A hyperparameter that scales the size of each gradient-descent weight update; too large causes divergence or oscillation, too small makes training slow or stuck.

Quantization

Reducing the numerical precision of model weights and activations, for example FP16 down to INT8 or FP8, to cut memory use and bandwidth at some cost in accuracy.

Data parallelism

A distributed training strategy that replicates the full model on every GPU and splits each batch across them, synchronizing gradients at every step.

Model parallelism

A distributed training strategy that splits a single model's layers or tensors across multiple GPUs, used when the model and its state do not fit on one device.

Model memory footprint

Approximate GPU memory for weights equals parameter count times bytes per parameter: FP32 uses 4 bytes, FP16 and BF16 use 2, and FP8 or INT8 use 1.

CPU versus GPU

CPUs have a few high-clock cores optimized for latency and serial work; GPUs have thousands of simpler cores optimized for throughput on highly parallel math.

Streaming multiprocessor

An NVIDIA GPU building block containing CUDA cores, Tensor Cores, registers, and shared memory; kernel thread blocks are scheduled onto these units for execution.

Tensor Core

Specialized GPU execution units that perform mixed-precision matrix multiply-accumulate operations in a single instruction, greatly accelerating deep-learning training and inference.

HBM

High Bandwidth Memory stacked beside the GPU die; its bandwidth, rather than capacity alone, often limits how fast large models can be served.

NVLink

A high-speed direct GPU-to-GPU interconnect far faster than PCIe; fourth-generation NVLink on H100 delivers about 900 GB/s bidirectional bandwidth per GPU.

NVSwitch

A switch chip that connects all GPUs in a server or pod fabric so every GPU pair can communicate at full NVLink bandwidth in an all-to-all topology.

PCIe Gen5 x16

A host interconnect lane configuration delivering about 64 GB/s per direction, roughly 128 GB/s bidirectional; it remains far slower than NVLink for peer GPU traffic.

MIG

Multi-Instance GPU partitions one physical GPU into as many as seven isolated instances, each with dedicated memory, cache, and compute units, for multi-tenant quality of service.

GPUDirect RDMA

A capability letting network adapters read and write GPU memory directly over an RDMA fabric, removing host-memory copies and lowering latency in multi-node training.

PUE

Power Usage Effectiveness, calculated as total facility power divided by IT equipment power; a value of 1.0 is ideal, and higher values mean more overhead on cooling and losses.

Fat-tree topology

A multi-tier network design of leaf and spine switches giving uniform, low-latency bandwidth between any two nodes; oversubscription above 1:1 can create congestion hotspots.

BlueField DPU

A data processing unit with its own Arm cores that offloads networking, storage, and security work from the host CPU and can accelerate AI cluster infrastructure.

nvidia-smi

A command-line utility that reports GPU inventory, utilization, memory use, temperature, power draw, and running processes at the moment it is executed.

DCGM

NVIDIA Data Center GPU Manager, a tool and API that collects GPU telemetry and health fields such as ECC errors, XID events, temperature, and throttle reasons.

XID error

A numeric driver-reported GPU event written to the kernel log; some codes are recoverable, while codes such as 79 (GPU fallen off the bus) usually need reset or replacement.

ECC error

An error flagged by Error Correcting Code memory; single-bit errors are corrected and logged, while uncorrectable double-bit errors can corrupt work and often require draining the GPU.

Availability formula

Availability equals MTBF divided by (MTBF + MTTR); raising mean time between failures or cutting mean time to repair both increase the fraction of time a service is usable.

Checkpoint interval

An optimal checkpoint period approximates the square root of twice the checkpoint write cost times MTBF, balancing lost work against checkpointing overhead.

NVIDIA GPU Operator

A Kubernetes operator that automates deployment of GPU drivers, the container runtime, the device plugin, DCGM monitoring, and MIG configuration across cluster nodes.

Gang scheduling

A scheduling policy that starts all parts of a multi-node job together or not at all, preventing deadlock and idle GPUs caused by partial allocations.

GPU utilization versus allocation

Allocation counts GPUs reserved by jobs, while utilization measures actual compute activity; high allocation with low utilization signals idle reserved capacity or poor job packing.

MTTR

Mean Time To Repair, the average time needed to restore a failed component or service; shortening it with spares, automation, and monitoring directly raises availability.