Exploratory Data Analysis
Exploratory Data Analysis
EDA in AWS ML often means using Amazon SageMaker Data Wrangler, Jupyter notebooks on SageMaker, or Athena/QuickSight for visualization. You must understand data profiling: summary statistics, missing values, outliers, and correlation. Know how to handle missing data (imputation vs. deletion), categorical encoding (one-hot, ordinal, target), and scaling (standardization, normalization). SageMaker Data Wrangler provides built-in transformations and quick visualizations. For large datasets, use SageMaker Processing jobs with Spark or scikit-learn. Also know SageMaker Clarify for bias detection and explainability, which is part of EDA for fairness.
Traps: Questions may ask about detecting bias before modeling; Clarify can compute pre-training bias metrics (e.g., class imbalance) and post-training metrics. Do not confuse Data Wrangler (visual data preparation) with Ground Truth (labeling). Another trap: using the entire dataset to compute scaling parameters leaks information; always fit on training only. Also, correlation does not imply causation, and high cardinality categorical variables can cause overfitting if one-hot encoded.
Worked example: You have a dataset with a missing age column and a categorical country column with 200 unique values. For age, impute with median; for country, use target encoding or group rare countries into "Other". Use Data Wrangler to visualize distributions and detect outliers. Rule of thumb: Always split data before any transformation that uses statistics; use pipelines to avoid leakage.