AI Fundamentals and Machine Learning
Domain scope
This domain tests whether you can frame a machine-learning problem, select a defensible metric, and explain why a model generalizes or fails. Expect scenarios with imbalanced labels, temporal data, and feature pipelines.
Core rules and concepts
- Define the label and the features that will actually be available at prediction time. A feature computed after the outcome is leakage.
- Bias–variance: underfitting shows high training and validation error; overfitting shows low training error and a widening validation gap.
- Regularization: L2 shrinks weights smoothly; L1 promotes sparsity. Dropout and early stopping are also regularizers.
- Metrics: precision = TP/(TP+FP); recall = TP/(TP+FN); F1 is their harmonic mean; ROC-AUC is threshold-independent; PR-AUC is more informative under severe class imbalance.
- Validation: stratified k-fold preserves class ratios; time-series data needs chronological splits; group splits prevent the same entity appearing in train and test.
Worked example
A fraud model is evaluated on 1,000 cases: 100 fraud, 900 legitimate. Confusion matrix: TP=70, FN=30, FP=45, TN=855. Accuracy = (70+855)/1,000 = 0.925, which looks strong. Precision = 70/(70+45) = 0.609; recall = 70/100 = 0.70; F1 = 2×0.609×0.70/(0.609+0.70) = 0.652. The high accuracy is driven by the majority class, so F1 or PR-AUC better reflects the fraud use case.
Traps that cost marks
- Tuning the threshold or hyperparameters on the test set; use a validation split and report the untouched test result.
- Applying target encoding or oversampling before the train/test split, which leaks label information.
- Choosing accuracy for a 1:100 imbalance, or ROC-AUC when the business cares about precision at a fixed alert budget.
- Randomly splitting time-ordered data, letting future information train the model.