NVIDIA NCA-AIIO key terms

A computing model made of layers of connected nodes, each with adjustable numeric weights, that learns to map inputs to outputs from examples rather than from hand-written rules.

A branch of machine learning that stacks many neural network layers so the model learns increasingly abstract features directly from raw data such as pixels, audio, or text.

The phase in which a model's weights are repeatedly adjusted by showing it data and minimizing a loss function, typically using forward passes and backpropagation over many batches.

Running an already-trained model on new inputs to produce predictions or generated text. It uses forward computation only and is usually optimized for low latency and high throughput.

One complete pass of the training algorithm over the entire training dataset. Models are usually trained for many epochs, with the data reshuffled between passes.

The number of samples processed together in one forward and backward pass. Larger batches use more GPU memory and can smooth gradients but may need learning-rate adjustment.

A positive scaling factor that controls how far weights move on each update step. Set too high, training diverges; set too low, training converges very slowly.

The method that computes how much each weight contributed to the training error by propagating gradients backward through the network layers, enabling weight updates.

A neural network architecture that applies self-attention across all positions of an input sequence in parallel, and is the basis for most modern language and multimodal models.

A very large transformer network trained on massive text corpora to predict the next token, giving it broad ability to generate, summarize, translate, and reason over language.

A small unit of text, roughly a word or word fragment, that a language model reads and produces. Inputs and outputs are counted in tokens for sizing and cost.

Continuing the training of a pretrained model on a smaller, task-specific dataset so its behavior adapts to a particular domain, style, or set of instructions.

Storing weights and activations in fewer bits, such as 8-bit integers or FP8, to shrink model memory and speed up inference, usually at a small cost in accuracy.

The numeric representation used for weights, activations, and gradients. FP32 is the baseline; FP16, BF16, and FP8 reduce memory use and let Tensor Cores compute faster.

A distributed training technique that splits a single layer's weight matrices across several GPUs, so each GPU holds a shard and results are combined with collective communication.

A processor containing thousands of arithmetic cores designed for parallel math. It is the main accelerator used to train and serve AI models, working alongside a host CPU.

NVIDIA's parallel computing platform and programming model that lets developers write code executed across thousands of GPU threads, and that underpins most AI frameworks.

Specialized execution units inside NVIDIA GPUs that perform small matrix multiply-and-accumulate operations in mixed precision, greatly accelerating the core math of deep learning.

High Bandwidth Memory: DRAM dies stacked and packaged next to the GPU die, connected by a very wide bus so the chip gets far more memory bandwidth than standard graphics memory.

A high-speed NVIDIA interconnect between GPUs that carries far more bandwidth than PCIe, making fast model-parallel training and low-latency GPU-to-GPU transfers practical.

A switch chip that links many GPUs through NVLink so that every GPU pair in a server or pod can communicate at full interconnect bandwidth without going through the CPU or PCIe.

A high-throughput, low-latency networking standard used to connect servers inside AI clusters, supporting remote direct memory access and the fabric needed for large distributed training jobs.

NVIDIA's integrated AI server product that bundles GPUs, CPUs, NVLink fabric, high-speed networking, and a software stack, validated together as a single unit for AI workloads.

A reference cluster design that joins many DGX systems with a high-bandwidth, low-latency network fabric so they act as one very large AI supercomputer.

Multi-Instance GPU: a feature that partitions one physical GPU into several isolated instances, each with its own memory slices and compute, so multiple users or jobs can share a device safely.

NVIDIA Collective Communications Library: optimized implementations of collective operations such as all-reduce and broadcast, used to synchronize gradients and tensors across GPUs in distributed training.

A data processing unit that pairs network interfaces with its own processor cores, offloading networking, storage, and security tasks from the host server so CPUs and GPUs stay focused on workloads.

A network layout in which nodes connect to leaf switches and those connect upward through spine switches, giving many equal-cost paths and predictable, largely non-blocking bandwidth between all servers.

A Kubernetes operator that automatically installs and manages NVIDIA drivers, container runtime hooks, device plugins, and monitoring components on GPU worker nodes.

NVIDIA Data Center GPU Manager: a toolset for monitoring GPU health and utilization, running diagnostic checks, and collecting telemetry across all nodes in a data center cluster.

A command-line utility that reports GPU inventory, memory usage, utilization, temperature, and running processes, and lets administrators set limits such as power or clock caps.

A numeric code written by the NVIDIA driver to the system log when a GPU hits a hardware or driver fault, used as the starting point for troubleshooting failures and deciding on remediation.

Applying new low-level software to GPUs, DPUs, baseboard controllers, or switches to fix defects and close security gaps, typically planned and rolled out carefully across a cluster to avoid outages.

NVIDIA software for provisioning, monitoring, and administering clusters of GPU nodes, covering image deployment, node health, and job-related cluster management tasks.

Automatic reduction of GPU clock frequency when temperatures or power draw exceed safe limits, which protects the hardware but lowers throughput and is a common cause of unexplained slowdowns.

NVIDIA's registry of GPU-optimized container images, pretrained models, and software development kits that teams pull into clusters to run AI workloads without building stacks from scratch.

An open-source container orchestration system that schedules workloads across a cluster of machines and manages their lifecycle, widely used to serve and run AI applications on GPU nodes.

An open-source workload manager that queues and schedules jobs on shared clusters, allocating nodes and GPUs to each job; common in high-performance computing and large training environments.