Machine Learning Concepts and Model Lifecycle
The ML lifecycle runs problem framing → data collection → feature engineering → training → evaluation → deployment → monitoring → retraining. Know where each activity sits and that the cycle is continuous, not linear.
Core concepts: overfitting (great on training data, poor on new data) versus underfitting (poor on both). Mitigate overfitting with more data, regularization, simpler models, or early stopping. The bias-variance tradeoff explains this tension. Always evaluate on data the model did not train on: a train/validation/test split or cross-validation. Choose metrics that match the problem: accuracy misleads on imbalanced classes; use precision, recall, F1, ROC-AUC, or RMSE/MAE for regression. Know the confusion matrix cold: precision = TP/(TP+FP), recall = TP/(TP+FN).
Data leakage is the most-tested trap: information from the target or from the future sneaks into training features (e.g., scaling before splitting, or using "account_closed_date" to predict churn). Concept drift is when input-output relationships change over time, degrading a deployed model; data drift is a change in input distribution. Both require monitoring and retraining.
Traps: questions implying the test set can be used for tuning (that makes it a validation set), or that a higher-accuracy model is always preferable regardless of class balance or cost of errors.
Worked example: a fraud model has 99% accuracy because 1% of transactions are fraud - it predicts "not fraud" always. Recall is 0. The right metrics are precision/recall and a cost-weighted threshold, not accuracy.
Rule of thumb: if a feature would not be available at prediction time in production, it is leakage.