Explore Data and Train Models
Exploratory analysis that actually matters
Exploration exists to find problems before training: missingness patterns, outliers, class imbalance, skewed distributions, correlated features, and label noise. Know summary statistics, distribution plots, and correlation analysis conceptually. Understand why missingness can be informative and the difference between dropping, imputing with a statistic, and imputing with a learned model. Recognize that imputation statistics must be fit on training data only.
Training and evaluation discipline
Candidates should understand splitting strategies: random holdout for i.i.d. data, stratified splits for imbalance, grouped splits when entities repeat, and time-based splits for temporal data. Know cross-validation and its variants, and why k-fold on time series leaks future information. Understand regularization (L1 for sparsity, L2 for smoothness), early stopping, and hyperparameter search methods from grid to random to Bayesian.
Traps to watch for
- Reporting cross-validation scores as final performance without a held-out test set.
- Scaling or encoding before splitting, leaking test statistics into training.
- Using random splits on time-ordered data.
- Comparing models trained on different feature sets or different data versions.
- Overfitting the validation set through excessive repeated tuning.
Worked example
With 2% positive class and 10,000 rows, a random split may put very few positives in validation. Use a stratified split so the positive rate is preserved, report precision and recall at a chosen threshold rather than accuracy, and reserve a final untouched test set for one evaluation.
Rule of thumb
Every transformation that learns a parameter (mean, scale, encoder vocabulary) is part of the model and must be fit inside the training fold.