Data Analysis
What this domain tests
This domain probes whether you can prepare, summarize, and validate data before it reaches a model: splits, leakage, imbalance, missing values, descriptive statistics, and correlation. The exam favors practical judgment over formula recall.
Core rules and concepts
- Split before fitting any transform. Scaling, imputation, and target encoding learned on the full dataset leak test information into training.
- Stratified splits preserve class proportions; use them for imbalanced classification and small datasets.
- Accuracy is misleading under imbalance. Report precision, recall, F1, and the confusion matrix.
- Correlation measures linear association, not causation, and is sensitive to outliers.
- Missing data mechanisms differ: MCAR, MAR, and MNAR. Mean imputation shrinks variance and is only safe when missingness is benign.
Worked example
A fraud model scores 1,000 transactions: 20 fraudulent and 980 legitimate. It predicts 'legitimate' for every row. Accuracy is 980 / 1,000 = 98%, which looks strong. Fraud recall is 0 / 20 = 0, and precision is undefined because no positive predictions were made. The confusion matrix (TP=0, FN=20, TN=980, FP=0) exposes the failure that accuracy hid. A useful baseline must beat the majority-class rate on a metric that weights the minority class, such as recall at a fixed false-positive budget.
Traps that cost marks
- Normalizing or imputing before splitting, then reporting inflated test scores.
- Quoting accuracy on a 99:1 dataset.
- Treating a high correlation as evidence of causation.
- Dropping rows with missing values without checking whether missingness correlates with the label.
- Computing statistics on the test set while tuning.