IA MLO flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

MLOps definition

A set of engineering practices that unify ML system development (Dev) and operations (Ops) to automate and reliably manage the full ML lifecycle, from data preparation through deployment and monitoring, at production scale.

Continuous training (CT)

An MLOps practice where the pipeline automatically retrains and re-evaluates models when new data arrives or a trigger fires, then promotes the model only if it passes defined quality gates.

Training-serving skew

A mismatch between how features are computed during training and during inference, often from separate code paths, causing production accuracy to fall below offline evaluation results.

Point-in-time correct join

A feature retrieval method that joins each training label only to feature values known before the prediction timestamp, preventing future information from leaking into training data.

Feature store

A system providing a curated, versioned feature repository with an offline store for training and an online store for low-latency serving, keeping both consistent from one definition.

Model registry

A versioned catalog of trained model artifacts with metadata, lineage, and lifecycle stages such as staging and production, used to control which model is deployed and to enable rollback.

Experiment tracking

Logging of parameters, metrics, artifacts, code version, and environment for each training run so results can be compared, reproduced, and audited later.

Shadow deployment

A release pattern where the new model receives a copy of live traffic and its predictions are logged but not served, allowing risk-free comparison before promotion.

Canary release

A deployment pattern routing a small percentage of live traffic to the new model, with automated monitoring and rollback if error or quality metrics degrade.

Blue-green deployment

A pattern maintaining two identical environments; traffic switches from the old (blue) to the new (green) version at once, and rollback is a fast switch back.

Data drift

A change in the distribution of model inputs P(X) versus the training distribution, detectable with statistics such as population stability index or Kolmogorov-Smirnov tests.

Concept drift

A change over time in the relationship between inputs and target, P(y|X), so the model's learned mapping becomes stale even if input distributions look unchanged.

Model lineage

Recorded provenance linking a deployed model to its training data version, code commit, hyperparameters, and parent artifacts, enabling audits and root-cause analysis.

Golden dataset

A fixed, curated evaluation set with trusted labels used as a regression gate to detect quality degradation before a model is promoted to production.

Model packaging

Bundling trained weights with preprocessing code, dependencies, and a defined inference interface into an immutable artifact such as a container image or serialized model file.

Immutable model artifact

A versioned, never-overwritten model bundle identified by a unique hash or tag, ensuring any deployed version can be reproduced and restored during rollback.

MLOps maturity level 0

A manual process where data preparation, training, and deployment are script-driven with no automated pipeline, continuous integration, or continuous delivery of models.

MLOps maturity level 1

Automation of the ML pipeline so that training and validation run automatically on new data, enabling continuous training, though deployment may still be manual.

MLOps maturity level 2

Full CI/CD plus continuous training: code, data, or pipeline changes automatically trigger build, test, training, evaluation, and deployment with quality gates.

Data versioning

Tracking dataset snapshots by content hash or a tool such as DVC, so each model run references an exact, restorable data state rather than a mutable path.

Pipeline component

A containerized, reusable step in an ML pipeline (for example data validation or training) with declared inputs and outputs, allowing steps to be orchestrated and cached.

Model serving endpoint

An API exposing a deployed model for online inference, with concerns such as request batching, autoscaling, latency budgets, and payload schema validation.

Batch inference

Offline scoring of a large dataset on a schedule, suitable when predictions are not needed in real time and throughput matters more than per-request latency.

Online inference

Real-time prediction served per request with strict latency limits, requiring a low-latency feature lookup and a warm, scalable model server.

Model quality gate

An automated check comparing a candidate model against thresholds and the current production baseline on a golden dataset before promotion is allowed.

Model rollback

Restoring a previously validated model artifact to production after detecting degradation, made fast by immutable artifacts and registry stage history.

Retraining trigger

A predefined condition that starts retraining, such as a schedule, a drift threshold breach, or a drop in monitored performance below a set limit.

Reproducibility manifest

A record pinning code commit, data version hash, environment image digest, hyperparameters, and random seeds so a training run can be exactly recreated.

Prediction drift

A shift in the distribution of model outputs over time, which can signal input changes or stale models even when ground-truth labels are unavailable.

Model card

A short document describing a model's intended use, training data, evaluation metrics, and known limitations, supporting governance and informed downstream use.

Pipeline orchestration

A scheduler that runs pipeline components in dependency order with retries, caching, and artifact passing, for example Kubeflow Pipelines, Airflow, or Vertex AI Pipelines.

A/B test for models

A controlled online experiment randomly splitting traffic between model versions and comparing a business or quality metric with statistical significance before full rollout.