Exploratory Data Analysis: Finding Problems Before Modeling
What EDA is actually for
EDA questions ask you to diagnose a dataset and choose the correction that will not poison the model.
Core checks. Missing values (rate, pattern, whether missingness is informative), outliers (domain-invalid vs merely extreme), duplicates, target leakage, class imbalance, cardinality of categoricals, skew/transform needs, correlation and multicollinearity, and, for time series, trend, seasonality, and stationarity. Visual inspection matters: distributions, box plots, scatter matrices, and correlation heatmaps catch what summary statistics hide.
Tools. SageMaker Data Wrangler for interactive profiling, built-in transformation templates, and quick charts; Athena or Redshift queries for aggregate statistics at scale; QuickSight for dashboards; SageMaker Processing jobs for repeatable large-scale cleaning; pandas/matplotlib in a notebook for small samples.
Reduction. PCA is linear, maximizes variance, and produces a reusable transform; t-SNE and UMAP are nonlinear, for visualization only, and cannot transform new points.
Traps. Leakage is the classic: normalizing, imputing, or oversampling before the train/test split lets test information into training, and similar leakage arises from features computed using the target or from duplicate rows split across folds. Accuracy on imbalanced data is meaningless. Data Wrangler is an exploration tool, not a scheduled production pipeline (that is Glue or Processing). Dropping rows with missing values can silently remove an entire class.
Worked example. A fraud dataset has 1% positives. A model that predicts "not fraud" scores 99% accuracy and is useless. Check the confusion matrix, then report precision, recall, F1, and PR-AUC; apply SMOTE or class weighting inside cross-validation folds only; and tune the decision threshold against the business cost of false negatives.
Rule of thumb. Split first, learn transformations on the training fold, apply them to validation and test.