Unsupervised Learning
What to know
Topics include k-means (objective, initialization sensitivity, choosing k via elbow or silhouette), hierarchical clustering (linkage criteria), DBSCAN (density, eps, minPts, noise handling), Gaussian mixture models and the EM algorithm (soft assignments, covariance structure), PCA (variance maximization, orthogonal components, explained variance, standardization prerequisite), t-SNE and UMAP (visualization only, no global distances), and anomaly detection. Also expect dimensionality-reduction tradeoffs and evaluation without labels (silhouette, Davies-Bouldin).
Traps
- Using t-SNE or UMAP distances as if globally meaningful - they preserve local neighborhoods only.
- Running k-means without scaling, letting large-range features dominate distance.
- Believing k-means handles non-convex clusters or noise; it assumes spherical, equal-variance clusters.
- Choosing PCA components by a fixed count rather than explained-variance threshold.
- Treating GMM hard assignments as equivalent to EM's soft responsibilities.
Rule of thumb
Standardize before PCA or k-means; use silhouette to compare clusterings; use DBSCAN when clusters are irregular and outliers must be isolated.
Worked example
You retain enough PCA components to reach 95% cumulative explained variance. If the first three eigenvalues are 4, 2, and 1 out of a total variance of 10, the cumulative is 70% - you need more components. Always sum the sorted eigenvalues cumulatively.