DE MLO key terms

A single execution of a training or evaluation script, identified by a unique ID and recording its parameters, metrics, artifacts, and code version so results can be compared later.

Structured facts attached to a run, such as hyperparameters, dataset identifiers, git commit, environment, and tags, used to filter, group, and audit experiments.

Recording scalar or series values over time during training, such as loss and accuracy per step, so curves can be compared across runs and checked for convergence.

Any file produced or consumed by a run, such as a model checkpoint, dataset snapshot, or evaluation report, stored with a content hash and version for traceability.

The recorded graph of which inputs produced which outputs, linking datasets, code, runs, and model versions so any deployed model can be traced back to its origins.

A central catalog that stores model versions with their metadata and stage labels, supporting promotion, rollback, and controlled handoff between training and serving teams.

An immutable snapshot of a trained model's weights and configuration, identified by a unique number or hash so results and deployments can be referenced unambiguously.

Moving a model version through lifecycle labels such as development, staging, production, or archived, with gates and approvals required before each promotion step.

Reverting a serving deployment to a previously validated model version when the new one degrades quality or fails health checks, restoring service quickly.

A saved copy of model weights and optimizer state at a training step, enabling resumption after failure and selection of the best-performing iteration.

A configuration value set before training, such as learning rate, batch size, or number of layers, that governs the learning process rather than being learned from data.

Systematically searching combinations of configuration values, using grid, random, or Bayesian methods, to improve a validation metric within a compute budget.

A rule that changes the learning rate during training, such as warmup followed by decay, to improve stability and final convergence quality.

An optimization procedure that iteratively updates model parameters in the direction that reduces the loss, using gradients computed on batches of training data.

The number of training examples processed before one parameter update, trading off memory use, throughput, and the noise level of gradient estimates.

A condition where a model fits training data noise so closely that it generalizes poorly to unseen data, visible as a widening gap between training and validation metrics.

Techniques such as weight decay, dropout, or data augmentation that constrain model complexity to reduce overfitting and improve performance on unseen examples.

Halting training when a validation metric stops improving for a set number of epochs, preventing overfitting and saving compute while preserving the best checkpoint.

A held-out portion of labeled data used to tune hyperparameters and select models, kept separate from training and final test data to avoid optimistic bias.

Accidental inclusion of information from the target or test period in training features, producing unrealistically high evaluation scores that collapse in production.

Creating and transforming input variables, through scaling, encoding, aggregation, or domain-derived ratios, so a model can learn patterns more effectively.

Starting from a model pretrained on a large related dataset and fine-tuning it on a smaller target dataset, reducing data and compute requirements.

Further training of a pretrained model on task-specific data, often with a smaller learning rate, to adapt its learned representations to a new objective.

Running a trained model behind an interface that accepts requests and returns predictions, with concerns for latency, throughput, and availability.

A network address and API, often HTTP or gRPC, that exposes a deployed model for real-time prediction requests from applications.

Scoring large volumes of records on a schedule using a deployed model, trading immediate response for higher throughput and lower per-record cost.

The time between sending a prediction request and receiving the response, a key service-level measure for interactive applications and autoscaling decisions.

The number of prediction requests or records a serving system processes per unit time, constrained by hardware, batching, and model size.

Releasing a new model version to a small fraction of live traffic first, comparing error rates and metrics before widening exposure or rolling back.

Running a candidate model in parallel with the live model on real requests without returning its output, to compare behavior safely before promotion.

A controlled experiment splitting traffic between model variants and comparing a business or quality metric to decide which version performs better.

A mismatch between how features are computed during training and during live prediction, causing production accuracy to fall below offline evaluation.

A shared system that computes and stores feature values consistently for both training pipelines and online serving, reducing skew and duplicated logic.

Continuous tracking of a deployed model's inputs, outputs, errors, and latency to detect degradation and trigger investigation or retraining.

A decline in predictive performance over time as the relationship between inputs and target changes in the real world after deployment.

A change in the statistical distribution of input features compared with the training data, which may or may not yet affect prediction quality.

A predefined condition, such as drift score or accuracy drop exceeding a threshold, that automatically starts a new training cycle.

A configured limit on a monitored signal that, when crossed, notifies operators so they can investigate before users are significantly affected.

A complete record of code commit, dependency versions, data snapshot, random seeds, and hardware needed to recreate a training run exactly.

A value initializing pseudo-random number generators so that data shuffling, weight initialization, and sampling repeat identically across runs.

A short document describing a model's intended use, training data, evaluation results, limitations, and ethical considerations for downstream consumers.

The documented origin and processing history of a dataset, including sources, transformations, and access controls, supporting audit and compliance review.

Rules assigning permissions so that only authorized users and services can read, modify, or deploy models, data, and experiment records.