Implement Machine Learning Model Lifecycle and Operations
Lifecycle questions pick up where the model finishes training. The spine is: run a training pipeline → evaluate metrics → pass a quality gate → register a versioned model with lineage → deploy → monitor → retrain or roll back. Each arrow is usually a separate exam question.
Deployment choice is the most common fork. Managed online endpoints serve real-time requests, autoscale on rules or metrics, and let you split traffic between deployments for blue-green or canary rollout; they bill while deployed, so scale-to-zero or delete idle ones. Batch endpoints score a data asset on a schedule - pick them when latency is measured in minutes or hours. Keep the previous deployment alive until the new one proves healthy, then shift traffic; rollback is a traffic-percentage change, not a redeploy.
Monitoring covers data drift, prediction drift, and feature-attribution drift, plus operational signals (latency, error rate) in Application Insights. Retraining triggers are scheduled, drift-alert-driven, or data-volume-driven; the exam likes drift plus a human approval gate rather than fully automatic promotion. Watch for training-serving skew - the model trained on one feature transformation but serves another.
Traps. Assuming a newly registered model is deployed automatically. Confusing model registration (metadata + artifact) with model deployment (endpoint + compute). Treating drift as proof the model is wrong - drift is a signal to investigate, not an automatic retrain. Ignoring that a gate must be a pipeline step that can fail.
Worked example: canary release - deploy new version at 10% traffic, compare latency and prediction distribution for a day, promote to 50%, then 100%; if error rate breaches threshold, set traffic back to 0% and investigate.