IA MLO cheat sheet

MLOps Engineering Practice: Lifecycle and Deployment

• ML lifecycle stages: problem framing → data collection/validation → feature engineering → training/experimentation → evaluation → packaging → deployment → monitoring → retraining; each stage produces a versioned artifact (dataset, feature set, model, config).

• Three levels of MLOps maturity: manual (Level 0, scripts and notebooks, no CI/CD), automated pipeline (Level 1, continuous training on new data), full CI/CD + CT (Level 2, automatic pipeline triggers on data/schema/code change).

• CI for ML extends software CI with data validation, model validation, and pipeline-component tests; CD deploys the pipeline (not just a model artifact) so the same pipeline can retrain and redeploy.

• Reproducibility requires pinning four things: code commit, data snapshot/version (hash or DVC pointer), environment (container image digest, dependency lockfile), and hyperparameters/random seeds.

• Experiment tracking records params, metrics, artifacts, and lineage per run; a model registry stores stage transitions (None → Staging → Production → Archived) and is the single source of truth for which artifact serves traffic.

• Feature store separates offline (batch training, e.g., Parquet/warehouse) from online (low-latency serving, e.g., Redis) stores; training-serving skew arises when the same feature is computed by different code paths - point-in-time correct joins prevent label leakage.

• Deployment patterns: shadow (mirror traffic, no user impact), canary (route a small % of traffic, compare metrics), blue-green (two full environments, instant switch/rollback), A/B (statistical comparison of business metrics), and batch/offline scoring for non-realtime needs.

• Model monitoring tracks data drift (input distribution change, e.g., PSI, KS test), concept drift (P(y|X) change, detected via performance decay), prediction drift, and operational signals (latency, error rate, throughput); alerts require a defined retraining trigger.

• Rollback and safety: keep the previous model artifact immutable and deployable, gate promotion on automated evaluation against a golden dataset plus online guardrail metrics, and record model lineage so any production prediction can be traced to data, code, and config.