Machine Learning Algorithms
The Machine Learning Algorithms domain tests whether you can match an algorithm to a problem type, interpret its hyperparameters, and choose an evaluation metric that reflects the real cost of errors. Questions commonly present a business scenario and ask which algorithm or metric is appropriate.
Algorithm families
- Linear regression fits a weighted sum of features to a continuous target by least squares. Logistic regression applies a sigmoid to a linear score and outputs a probability for classification.
- k-nearest neighbours classifies by majority vote of the k closest training points; features must be scaled because distance dominates.
- Decision trees split data on feature thresholds to reduce impurity (Gini or entropy); deep trees overfit, so depth and minimum samples per leaf are tuned. Random forests average many decorrelated trees (bagging); gradient boosting builds trees sequentially to correct residuals.
- Support vector machines maximize the margin between classes and can use kernels for non-linear boundaries. Naive Bayes applies Bayes' theorem assuming conditional independence of features.
- k-means partitions points into k clusters around centroids; choose k with domain knowledge or the elbow/silhouette method. Hierarchical clustering builds a dendrogram.
- Evaluation: a confusion matrix gives TP, FP, FN, TN. Precision = TP/(TP+FP); recall = TP/(TP+FN); F1 is their harmonic mean. Accuracy is misleading when classes are imbalanced. Regression uses MAE, RMSE, and R-squared. Use k-fold cross-validation and tune hyperparameters with grid or random search on training folds.
Worked example
A fraud detector sees 1,000 transactions, 10 fraudulent. It flags 13 transactions, catching 8 frauds. TP=8, FP=5, FN=2. Precision = 8/13 = 0.62; recall = 8/10 = 0.80; F1 = 2×0.62×0.80/(0.62+0.80) ≈ 0.70. A model that predicts 'not fraud' for everything scores 99% accuracy but recall 0 and is useless. Because missing fraud is costly, the team favours recall, accepting more false alerts for human review.
Traps
Do not report accuracy alone on skewed data. Do not tune hyperparameters on the test set. Do not forget scaling for k-NN, SVM, and k-means. Do not use regression for a categorical target. Do not assume a higher k in k-NN always improves fit; it smooths the boundary and can underfit.