NVIDIA GenAI LLM key terms

A neural network architecture that processes all positions of a sequence in parallel using stacked attention and feed-forward layers, forming the basis of most modern large language models.

A mechanism that lets each token in a sequence weigh the relevance of every other token and blend their representations, capturing context without recurrence.

The step that splits raw text into smaller units called tokens, often subwords, which are then mapped to integer IDs the model can process.

A dense numeric vector representing a token, sentence, or document so that semantically similar items sit close together in vector space.

The maximum number of tokens a model can attend to at once, covering both the input prompt and the generated output.

Stored key and value tensors from earlier tokens that let a model generate text step by step without recomputing attention over the whole prefix.

Continuing to train a pretrained model on a smaller, task-specific dataset so it adapts its behavior to a particular domain or objective.

A parameter-efficient fine-tuning method that freezes the base weights and trains small low-rank adapter matrices, cutting memory and storage costs.

Reducing the numerical precision of model weights and activations, for example from 16-bit to 8-bit or 4-bit, to shrink memory use and speed up inference.

Training a smaller student model to imitate the outputs or internal signals of a larger teacher model, preserving much of the quality at lower serving cost.

A metric reporting how surprised a language model is by a text; lower values mean the model assigns higher probability to the actual next tokens.

A sampling parameter that scales the model's output distribution; low values make generation more deterministic, while high values make it more random and diverse.

The production phase in which a trained model processes new inputs and returns predictions or generated text, as opposed to the training phase.

When a model memorizes training data too closely, performing well on examples it has seen but poorly on new, unseen data.

An optimization procedure that repeatedly nudges model parameters in the direction that reduces the loss, using gradients computed on batches of data.

An approach that retrieves relevant documents from an external store and inserts them into the prompt so the model can answer with grounded, current information.

Partitioning data into subsets for fitting parameters, tuning model choices, and a final unbiased performance estimate, keeping the test set untouched until the end.

When information from the target or the test set sneaks into training features, producing evaluation scores that look strong but do not generalize.

A dataset condition where one label is far more common than others, which can push models toward the majority class and hide poor minority-class performance.

Creating modified copies of training examples, such as rotations or paraphrases, to increase data variety and reduce overfitting.

Selecting samples so that each subgroup keeps the same proportion as in the full dataset, commonly used to preserve class balance across splits.

A controlled experiment that randomly assigns users to a control and a treatment group, then compares outcomes to measure the effect of a change.

The default claim that there is no real difference or effect between groups; statistical tests assess how compatible the observed data are with it.

The probability of observing results at least as extreme as the data if the null hypothesis were true; small values suggest the effect is unlikely to be chance.

The chance an experiment correctly detects a real effect of a given size, driven by sample size, effect size, and the chosen significance threshold.

The smallest change an experiment is designed to reliably detect; it determines how much traffic or how many samples the test needs.

A range around an estimate that, across repeated sampling, would contain the true value a stated percentage of the time.

A secondary metric tracked in an experiment to catch harmful side effects, such as crashes or added latency, even when the primary metric improves.

Continuous integration and continuous delivery: automated pipelines that build, test, and deploy code changes frequently and reliably.

Packaging an application with its dependencies into an isolated, portable image so it runs identically across development, testing, and production environments.

An automated check that exercises a small, isolated piece of code and asserts its expected behavior, catching regressions early in development.

The infrastructure and APIs that host a trained model and handle prediction requests in production, including scaling, monitoring, and version management.

An attack where adversarial text in user input or retrieved content hijacks the model's instructions, causing it to ignore or override its intended behavior.

Confidently stated model output that is factually wrong or unsupported by any source, often arising when the model lacks the needed knowledge.

Systematic unfairness in model outputs that disadvantages particular groups, usually traceable to skewed data, labels, or design choices.

A quantitative measure, such as equal error rates or demographic parity, used to check whether model outcomes are distributed equitably across groups.

Adversarial testing in which people deliberately try to make a model fail, leak data, or produce harmful content in order to find and fix weaknesses.

Personally identifiable information: data that can single out an individual, such as names or government IDs, which must be protected in training data and prompts.