Data Preparation and Feature Engineering
What to know
Questions probe whether you understand why each preprocessing step exists and when it can leak information. Cover: missing-data mechanisms (MCAR, MAR, MNAR) and imputation strategies; scaling (standardization, min-max, robust scaling) and which models require it; encoding categoricals (one-hot, ordinal, target/label encoding and its leakage risk); feature transformations (log, Box-Cox, binning); handling outliers; class imbalance (resampling, class weights, SMOTE); and the cardinal rule that all fitted transformations must be learned on training data only and applied to validation/test.
Traps
- Fitting a scaler or imputer on the full dataset before splitting - leakage that inflates test scores.
- Using target encoding without out-of-fold computation, leaking the label.
- Assuming tree-based models need feature scaling (they don't) while distance- and gradient-based models do.
- Dropping rows with missing values when the mechanism is MAR and imputation is safer.
- Applying SMOTE before the train/test split, synthesizing points across the boundary.
Rule of thumb
Any step that computes a statistic from data (mean, median, target rate) belongs inside the training pipeline and must be refit per fold in cross-validation.
Worked example
Target encoding a high-cardinality "city" feature: compute each city's mean target using only the training fold, optionally smoothed toward the global mean with m-estimates for rare categories, then map it onto validation data. Computing it on the whole dataset first is the leak examiners love to test.