300-640 DCAI key terms

A documented, repeatable design that specifies compute, network, storage, power, and cooling tiers for a target AI workload class, so that sizing, ordering, and deployment follow a validated blueprint rather than ad hoc builds.

The distinction between workloads that iterate over large datasets to update model weights and workloads that only run forward passes on live requests; they differ sharply in compute type, latency tolerance, and traffic pattern.

The electrical load a single rack must support, usually expressed in kilowatts; GPU servers push this far above traditional enterprise values and drive decisions on power distribution, cooling method, and floor layout.

The ratio of total facility energy to the energy delivered to IT equipment. A value near 1.0 indicates little overhead from cooling and distribution losses, and it is the common benchmark for data center efficiency claims.

The high-bandwidth, low-latency network that interconnects accelerators so they can exchange gradients and activations during distributed training, typically built from dedicated switching tiers separate from general-purpose data traffic.

Splitting a single model's layers or tensors across multiple accelerators so that a model too large for one device's memory can still be trained or served, at the cost of frequent cross-device communication.

Replicating the full model on each accelerator and dividing the input batch among them, then synchronizing gradients across replicas; simple to implement but sensitive to collective-communication efficiency.

A collective operation in which every participant ends up holding the same reduced result, such as a summed gradient. It dominates training traffic and is the primary reason interconnect bandwidth and latency matter so much.

Software such as NCCL that implements all-reduce, broadcast, and all-gather across accelerators, choosing algorithms and transport paths. Its tuning and version compatibility directly affect achievable training throughput.

A family of capabilities that let accelerators read and write host memory, storage, or peer device memory directly, bypassing CPU copies. It reduces latency and CPU overhead for data loading and inter-node communication.

A transport that moves data between host memories without involving the remote processor, giving high throughput with low CPU load. It underpins lossless, high-performance AI fabrics and distributed storage access.

An Ethernet-based RDMA implementation that encapsulates RDMA packets in routable UDP/IP, allowing a lossless fabric to be built on standard Ethernet switches rather than a dedicated specialized network.

A hop-by-hop Ethernet mechanism that pauses only a specific traffic class when a buffer fills, preventing packet drops for lossless flows while leaving other classes unaffected. Misconfiguration can cause pause storms.

A marking scheme in which switches tag packets before buffers overflow so endpoints can slow down, reducing reliance on pause frames and smoothing congestion across a large fabric.

Traffic between servers inside the data center, such as gradient exchange, storage reads, and service-to-service calls. AI workloads make it dominant, inverting the traditional north-south traffic assumption.

A two-tier network design in which every leaf switch connects to every spine switch, giving predictable, uniform hop counts between any pair of endpoints and easy horizontal scaling of bandwidth.

The relationship between aggregate edge port bandwidth and uplink bandwidth toward the core. A ratio of 1:1 is non-blocking; higher ratios save cost but can bottleneck collective operations.

A network provisioned so that every endpoint can communicate with every other at full line rate simultaneously, avoiding contention that would otherwise stall synchronized distributed training steps.

A long-lived, high-volume flow such as a storage read or a large data transfer. It can monopolize a link and degrade many small latency-sensitive flows unless queuing or load balancing mitigates it.

A storage system that spreads file data and metadata across many servers so aggregate throughput scales with node count, matching the concurrent read demands of many accelerators loading training data.

A flat namespace that stores data as immutable objects with metadata and keys, accessed over HTTP APIs. It scales cheaply for datasets and archives but has higher per-request latency than file or block access.

An architecture that keeps data in open file formats on cheap storage while adding table metadata and transaction guarantees, supporting both analytics and machine learning from a single governed copy.

A periodic snapshot of model weights and optimizer state written to durable storage so training can resume after failure. Checkpoint size and write bandwidth often dictate storage sizing for large jobs.

Keeping compute close to the data it processes, either physically or through caching and scheduling, to avoid network bottlenecks and idle accelerators waiting on input pipelines.

A managed repository that serves curated, versioned model inputs consistently to both training pipelines and online inference, reducing duplicated engineering and preventing training-serving skew.

A recorded trace of where data came from and how it was transformed, used to reproduce model results, audit compliance, and debug quality problems discovered after deployment.

A versioned catalog of trained models with metadata, lineage links, and stage labels, enabling controlled promotion from experimentation to production and reliable rollback to a prior artifact.

An automated sequence covering data preparation, training, evaluation, packaging, and deployment, with repeatable steps and approvals so models can be retrained and released without manual assembly.

Defining servers, networks, and platform services in version-controlled declarative files so environments can be recreated identically, reviewed like software, and rolled back after faulty changes.

The gradual divergence between the deployed state of systems and their declared desired state, often caused by manual fixes; it undermines reproducibility and can silently break GPU or fabric settings.

An operating model in which administrators declare desired outcomes such as policy or reachability, and the controller translates them into device configuration and continuously verifies compliance.

Streamed measurements from infrastructure and workloads, such as link errors, GPU temperature, and queue depth, used for capacity planning, anomaly detection, and proving service-level objectives are met.

Software that queues, places, and preempts workloads across a shared cluster, deciding which accelerators and network resources each job receives and how long it may hold them.

An allocation policy that starts a distributed job only when all of its requested workers can run at once, preventing partial allocations that would hold expensive accelerators idle while waiting.

Technical separation ensuring one team's workloads cannot read another's data or consume its resources, enforced through namespaces, network policy, identity controls, and dedicated hardware pools.

Sharing one AI platform among several teams or customers while preserving fairness, privacy, and predictable performance, usually through quota, scheduling policy, and network segmentation.

Using hardware-isolated enclaves so data and models remain encrypted in memory during processing, limiting exposure to privileged operators or a compromised host operating system.

The maximum acceptable duration to restore a service after failure. For training platforms it drives checkpoint frequency, spare capacity, and whether jobs restart from scratch or resume.

The maximum acceptable amount of data loss measured in time. It determines how often checkpoints or replicas must be taken to keep lost work within agreed limits.

Removing heat with a circulating fluid, either to a cold plate on the processor or by immersing hardware, which handles densities that air cooling cannot and can lower total facility energy.

Releasing a new model or configuration to a small slice of traffic first, comparing error rates and latency against the baseline before widening the rollout or rolling back.