NVIDIA NCA-GENM flashcards

31 cards. Read the term and the definition here. The deck at the top of this page is optional.

Scaled dot-product attention

Attention computes softmax(QKᵀ/√d_k)V, so each output token is a weighted sum of value vectors whose weights come from query-key similarity scaled by the square root of head dimension.

LoRA adapter

Low-rank adaptation freezes the pretrained weight matrix and adds a trainable product BA of rank r, cutting trainable parameters by orders of magnitude and enabling swappable task adapters at inference.

Mixture-of-Experts routing

A gating network scores each token and dispatches it to its top-k expert feed-forward layers, so total parameters grow while compute per token stays near that of k experts.

Classifier-free guidance

Sampling mixes conditional and unconditional noise predictions as ε̂ = ε_uncond + s(ε_cond − ε_uncond). Scale s above 1 strengthens prompt adherence but reduces diversity and can oversaturate outputs.

Chinchilla scaling rule

Compute-optimal training allocates roughly 20 tokens per model parameter, with loss following power laws in parameters, data, and compute C ≈ 6ND; below this ratio models are undertrained.

F1 score

Harmonic mean of precision and recall, F1 = 2PR/(P+R). It penalizes models that obtain high precision by sacrificing recall, a failure mode plain accuracy hides on imbalanced datasets.

Perplexity

Exponentiated average negative log-likelihood per token, exp(−(1/N)Σ log p(x_i)). Lower is better, but values compare only between models sharing the same tokenizer and vocabulary.

KL divergence

D_KL(P‖Q) = Σ P(x) log(P(x)/Q(x)) measures the extra bits needed to encode distribution P using Q. It is non-negative and asymmetric, so the direction must be stated.

Data leakage

Information from the target or test set contaminating training, such as duplicate rows across splits or features computed after the label. It inflates validation scores that then collapse in production.

Cosine similarity

Dot product of two vectors divided by the product of their norms, ranging from −1 to 1. It ignores magnitude, which suits normalized text and image embeddings used in retrieval.

Random versus grid search

Grid search exhausts a fixed lattice and scales exponentially with dimensionality; random search samples independently and finds good values faster when only a few hyperparameters actually matter.

Ablation study

Removing or replacing one component and retraining under identical data, seeds, and compute budget, so any metric change is attributable to that component rather than to confounds.

Early stopping

Halt training when validation loss stops improving for a patience window, then restore the best checkpoint. It limits overfitting and saves GPU hours without altering the architecture.

Learning rate warmup

Linearly ramping the learning rate from near zero across the first few hundred to few thousand steps stabilizes transformer training and prevents early divergence from large gradient updates.

CLIP contrastive pretraining

Matched image-text pairs in a batch are positives and all other pairs are negatives; the InfoNCE objective pulls matched embeddings together and pushes mismatched ones apart into a shared space.

Cross-attention fusion

Queries from one modality attend to keys and values from another, letting text tokens select relevant image regions. It differs from self-attention, which mixes tokens within a single modality.

Log-mel spectrogram

Audio converted into time-frequency energy on a mel scale, typically 80 bins with 25 ms windows and 10 ms hop, producing the two-dimensional input consumed by speech encoders.

Visual token budget

Number of tokens an image or video contributes to a multimodal prompt, set by patch count and resolution. Large budgets crowd out text and raise latency and KV-cache memory.

Modality gap

Image and text embeddings trained contrastively still occupy separate regions of the shared space, so raw cross-modal similarity scores are not calibrated without additional alignment training.

KV cache

Stored key and value tensors for every processed token, avoiding recomputation during autoregressive decoding. Size scales linearly with sequence length, batch size, layers, and head dimension.

Arithmetic intensity

FLOPs performed per byte of memory moved. Compared against the GPU ops-to-bytes ratio, it predicts whether a kernel is memory-bound or compute-bound and guides fusion and batching choices.

Tensor parallelism

Splitting weight matrices column- or row-wise across GPUs so each device holds a shard; every layer needs an all-reduce, making it communication-heavy and best kept within one NVLink domain.

FlashAttention

An exact attention kernel that tiles Q, K, and V through on-chip SRAM and never writes the full N×N score matrix to HBM, reducing memory to linear and accelerating long sequences.

Speculative decoding

A small draft model proposes several tokens, and the target model verifies them in one forward pass, accepting the longest correct prefix while preserving the output distribution at lower latency.

TensorRT-LLM

NVIDIA library that compiles large language models into optimized TensorRT engines with in-flight batching, paged KV cache, and FP8 or INT4 quantization for low-latency GPU serving.

Safetensors format

A serialization format storing tensors with a JSON header plus raw buffers. It avoids pickle's arbitrary code execution risk and enables fast zero-copy loading of large checkpoints.

Exponential backoff retries

A retry policy that waits progressively longer, usually with jitter, between attempts after an API returns 429 or 5xx, preventing synchronized retry storms across many clients.

NeMo Guardrails

An open-source toolkit that wraps a language model with input, dialogue, and output rails written in Colang, blocking jailbreaks, off-topic answers, and PII leakage before responses reach users.

NIST AI Risk Management Framework

A voluntary US framework organized around Govern, Map, Measure, and Manage functions, giving organizations a lifecycle structure for identifying, assessing, and treating AI risks.

Disparate impact ratio

Ratio of favorable outcome rates between an unprivileged and a privileged group. A value below 0.8, the four-fifths rule, is commonly treated as evidence of adverse impact.

Model card

A short document reporting intended use, training data, evaluation results disaggregated by subgroup, and known limitations, supporting downstream auditing and informed deployment decisions.