Machine Learning Lifecycle and ML Workflows
The lifecycle
A widely used reference is CRISP-DM: business understanding → data understanding → data preparation → modelling → evaluation → deployment, and it is explicitly iterative, not a waterfall. Modern practice wraps this in MLOps: monitoring in production, drift detection, retraining, and versioning of data, code and models. Data and model form a feedback loop - production data feeds the next training cycle.
Test activities per stage
- Data understanding/preparation: data quality checks, completeness, label correctness, representativeness, leakage checks.
- Modelling: hyperparameter and architecture selection, cross-validation, comparison against a baseline.
- Evaluation: hold-out test set, business-metric validation, bias and robustness checks.
- Deployment: integration testing, shadow/canary release, A/B testing, monitoring and alerting.
The central trap: leakage
Anything learned from data - scaling parameters, imputation values, target encoding, feature selection, oversampling - must be fitted on the training fold only. Fitting before splitting leaks test information and inflates results. A test set reused for repeated tuning silently becomes a validation set and stops being an unbiased estimate.
Also: the lifecycle is a loop. A model validated at deployment is not validated forever; data drift invalidates yesterday's evidence.
Worked example
You split 100,000 rows, then standardise all features using the full dataset mean and standard deviation, then train. Trap: the scaler already saw the test rows. Correct order: split → fit scaler on train only → transform train and test with those parameters → train → evaluate once.
Exam traps
- Treating the ML lifecycle as linear and ending at deployment.
- Confusing validation set (for tuning) with test set (for final unbiased estimate).
- Forgetting that data and model versions must be traceable for reproducibility.