ML Engineering, Deployment, and MLOps
What to know
Questions cover the ML lifecycle: experiment tracking, reproducibility (seeds, versioned data and code), feature stores, model registries, CI/CD for models, containerization, serving patterns (batch, real-time, streaming), latency/throughput tradeoffs, model compression (quantization, pruning, distillation), and monitoring for data drift, concept drift, and performance decay. Also expect A/B testing, shadow deployment, canary rollouts, and rollback strategies.
Traps
- Monitoring only system metrics (CPU, latency) while ignoring input/prediction drift.
- Assuming a model that performed well offline will hold online without monitoring.
- Deploying a full retrain as a canary without a rollback path.
- Confusing data drift (input distribution shift) with concept drift (input-output relationship shift).
- Neglecting training-serving skew from inconsistent feature pipelines.
Rule of thumb
Log inputs and predictions from day one; alert on distribution shift and label-delayed performance; keep a versioned artifact and a tested rollback.
Worked example
A credit model's approval rate drops sharply though code is unchanged. Compare feature distributions against training baselines (e.g., PSI); a large shift signals data drift, prompting retraining or a guardrail - not a blind redeploy.