NCA Foundations flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Amdahl's Law

A formula giving maximum speedup when only part of a workload is parallelized: 1 divided by the sum of the serial fraction and the parallel fraction divided by its speedup.

Roofline Model

A performance model plotting attainable floating-point throughput against arithmetic intensity, bounded by peak compute and by memory bandwidth times intensity.

Arithmetic Intensity

The ratio of floating-point operations performed to bytes moved between memory and compute; high values indicate compute-bound kernels, low values memory-bound ones.

NVLink

A high-bandwidth direct GPU-to-GPU interconnect that bypasses PCIe, enabling faster peer transfers and collective communication in multi-GPU servers.

Warp

A group of 32 threads scheduled together on NVIDIA GPUs; threads in a warp execute the same instruction, diverging only when control flow differs.

Occupancy

The ratio of active warps resident on a streaming multiprocessor to the maximum supported, limited by registers, shared memory, and block size.

Memory Coalescing

The hardware merging of simultaneous accesses by threads in a warp into a minimal number of memory transactions, achieved when adjacent threads read adjacent addresses.

Shared Memory Bank Conflict

A serialization that occurs when threads in a warp access different addresses mapping to the same shared memory bank, multiplying the cost of the access.

Warp Divergence

The serialization of execution paths when threads within one warp take different branches, so each path executes while other threads idle.

CUDA Stream

An ordered queue of GPU work; operations in one stream execute in order, while separate streams may overlap to hide transfer and kernel latency.

cuDNN

A NVIDIA library of tuned deep learning primitives such as convolutions, pooling, and normalization that frameworks call to obtain optimized GPU kernels.

TensorRT

An inference optimizer and runtime that fuses layers, selects precision and kernels, and serializes a GPU-architecture-specific engine for low-latency deployment.

NCCL

A library implementing collective communication operations such as all-reduce and broadcast across multiple GPUs and nodes, selecting algorithms based on topology.

Compute Capability

A version number identifying a GPU's instruction set and feature set; code must be compiled for a capability supported by the target device.

Mixed Precision Training

Training that uses lower-precision formats such as FP16 or BF16 for most math while retaining FP32 master weights, often with loss scaling for stability.

Data Parallelism

A distributed training strategy that replicates the full model on each device, splits each batch, and averages gradients across replicas before the optimizer step.

Checkpoint

A saved snapshot of model weights and optimizer state written during training so that a job can resume after interruption without restarting from scratch.

Learning Rate Warmup

A schedule that starts with a small learning rate and increases it over initial steps, reducing instability when training with large batches or adaptive optimizers.

Validation Loss

Error measured on data withheld from training; rising validation loss while training loss falls is a standard signal of overfitting.

cuDF

A GPU dataframe library in the RAPIDS ecosystem offering pandas-like APIs so that filtering, joins, and aggregation execute in parallel on device memory.

Apache Arrow

A columnar in-memory data format with a standardized layout that enables zero-copy interchange between GPU dataframes and other analytics tools.

Dask-cuDF

A distributed extension that partitions GPU dataframes across workers, letting datasets larger than one device's memory be processed in parallel.

RAPIDS Accelerator for Spark

A plugin that replaces supported Spark SQL and DataFrame operations with GPU implementations, falling back to CPU when types or expressions are unsupported.

Post-Training Quantization

Reducing weight and activation precision after training, using calibration data to choose scales, which lowers memory and latency with modest accuracy loss.

Continuous Batching

A serving technique that admits and retires requests at each decoding step rather than waiting for a whole batch, improving throughput for variable-length generation.

Structured Pruning

Removing entire channels, filters, or attention heads so that resulting tensors shrink; unlike unstructured sparsity, it typically yields real latency reductions.

Knowledge Distillation

Training a smaller student model to match a larger teacher's output distribution, transferring accuracy into a model with lower inference cost.

Time to First Token

The latency between receiving a generation request and emitting its first output token, dominated by prompt processing and queueing delay.

DCGM

NVIDIA Data Center GPU Manager, a toolset for GPU health checks, telemetry collection, and integration with cluster monitoring systems such as Prometheus.

Xid Error

A numeric fault code reported by the NVIDIA driver when a GPU encounters an error; the code identifies the failure class and guides diagnosis.

GPU Out-of-Memory Error

A runtime failure when a device allocation cannot be satisfied; causes include oversized batches, memory leaks, fragmentation, or concurrent processes sharing the GPU.

Jetson

A family of power-efficient embedded modules combining an NVIDIA GPU with an ARM CPU, used for robotics, vision, and edge inference deployments.