IA MLO key terms

The set of engineering practices and cultural norms that keep machine learning systems reliable from data collection through training, deployment, and ongoing operation, combining software engineering, data engineering, and model management disciplines.

The end-to-end sequence a model passes through: problem framing, data preparation, experimentation, training, validation, deployment, monitoring, and eventual retirement or replacement. Each stage has distinct tooling, ownership, and quality gates.

Recording the inputs and outputs of each training run, such as hyperparameters, dataset versions, code commits, metrics, and artifacts, so results can be compared, audited, and reproduced later. Usually implemented with a metadata store and run identifiers.

A versioned catalog of trained model artifacts with metadata such as lineage, metrics, stage labels, and approval status. It acts as the handoff point between experimentation and deployment pipelines.

Assigning unique, immutable identifiers to each trained model artifact and its associated data and code state, so any deployed prediction can be traced back to an exact build and reproduce or roll back safely.

The ability to recreate a prior training result or prediction from recorded inputs: code revision, data snapshot, environment, random seeds, and configuration. It is a prerequisite for debugging regressions and for regulatory audit.

Tracking changes to datasets and their transformations over time with immutable snapshots or content hashes, so training runs can be tied to a precise data state and data changes can be reviewed like code changes.

A shared system that computes, stores, and serves feature values consistently for both offline training and online inference, reducing training-serving skew and letting teams reuse curated features across models.

A mismatch between how features or preprocessing are computed during training versus during live inference, which silently degrades predictions. Common causes are different code paths, stale batch data, and inconsistent time windows.

Automatically building and testing code changes on every commit, including data validation and model quality tests in ML repositories, so defects are caught before they reach shared branches or deployment artifacts.

Keeping every validated change in a deployable state and automating promotion through environments, so releasing a model or pipeline update is a routine, low-risk, repeatable action rather than a manual event.

Automatically retraining and revalidating models when triggered by schedules, new data, or detected performance drift, then promoting candidates that pass quality gates. It extends CI/CD to cover model artifacts, not just code.

An orchestrated, repeatable workflow of steps such as data extraction, validation, transformation, training, evaluation, and packaging. Pipelines make each stage testable and let the same definition run in development and production.

A scheduler and dependency manager that runs pipeline steps in the correct order, handles retries, parallelism, and failure alerts, and records run history. Examples include workflow DAG schedulers used to coordinate data and model jobs.

A graph of tasks with one-way dependencies and no cycles, used to express pipeline order. It lets an orchestrator determine which steps can run in parallel and which must wait for upstream outputs.

Exposing a trained model so applications can obtain predictions, typically through an API or embedded library. Serving concerns include latency, throughput, concurrency, input validation, and version routing.

Generating predictions synchronously per request while a user or system waits, which demands low latency and high availability. It usually requires a resident model server and careful capacity planning.

Scoring a large set of records on a schedule or on demand, writing predictions to storage for later use. It favors throughput and cost efficiency over per-request latency and suits non-interactive use cases.

The persisted output of training: weights, serialization format, preprocessing code or configuration, and metadata needed to load and run the model. Artifacts must be versioned and validated before deployment.

Bundling a model artifact with its runtime dependencies, preprocessing logic, and configuration into a portable unit such as a container image, so it behaves identically across environments.

Packaging an application and its libraries into an isolated image that runs consistently on any compatible host. For ML it pins framework and driver versions, avoiding environment drift between training and serving.

The process of releasing a trained model into an environment where it serves real traffic, including configuration, capacity, access control, and rollback plans. Deployment patterns determine how risk is contained.

Routing a small fraction of live traffic to a new model version while most traffic stays on the current one, then gradually increasing exposure if metrics stay healthy. It limits blast radius from a bad release.

Maintaining two identical environments, with only one receiving traffic at a time. A new version is deployed to the idle environment, verified, then traffic is switched over, enabling fast rollback by switching back.

Running a candidate model in parallel with the production model on the same live inputs but without affecting user-facing responses, so its predictions can be compared safely before any traffic is shifted.

Reverting a deployment to a previously known-good model or configuration when a release causes errors or quality regressions. Fast rollback depends on versioned artifacts and automated traffic switching.

Continuously observing a deployed model's inputs, outputs, latency, and business outcomes to detect failures, drift, or degradation. It closes the loop between production behavior and future retraining decisions.

A change in the statistical distribution of input features relative to the training data, without necessarily changing the input-output relationship. It can erode accuracy even when the underlying concept is stable.

A change over time in the true relationship between inputs and the target, so the same feature values now imply different outcomes. Labels are needed to detect it, making delayed feedback a key challenge.

An umbrella term for any degradation of a deployed model's predictive performance, whether caused by data drift, concept drift, upstream pipeline changes, or stale features. It typically triggers retraining or investigation.

The verified correct label or outcome used to evaluate predictions. In production it often arrives late or partially, so monitoring systems must join delayed labels back to earlier predictions.

The process of assigning target values to examples, whether by human annotation, rule-based extraction, or feedback signals. Label quality and consistency directly bound the achievable model performance.

Automated checks that incoming data meets expectations for schema, ranges, null rates, and distribution, run before training or scoring. Failures block bad data from silently corrupting models or predictions.

Creating, transforming, and selecting input variables that make patterns easier for a model to learn. In production these transformations must be versioned and shared between training and serving code.

Searching over configuration values that control the learning process, such as learning rate or tree depth, using validation performance to choose a setting. Tuning runs must be tracked to avoid unreproducible results.

Assessing a trained model against held-out data and task-appropriate metrics, including subgroup and robustness checks, to decide whether it is fit for deployment. Evaluation criteria should be defined before training begins.

An independent gate that confirms a candidate model meets accuracy, fairness, latency, and compliance requirements before release. It separates the team that builds a model from the team that approves it.

Randomly splitting live traffic between a control and a treatment model and comparing outcome metrics statistically. It measures real-world impact but requires sufficient sample size and careful guardrail metrics.

Techniques that explain why a model produced a given prediction or which inputs drive its behavior overall. It supports debugging, fairness review, and regulatory obligations in high-stakes domains.