EXIN BCS ML Award cheat sheet

Machine learning is the field of building systems that improve task performance by learning patterns from data rather than through explicitly coded rules.

Supervised learning maps labelled input-output pairs to a model that predicts outputs for unseen inputs; classification predicts discrete labels, regression predicts continuous values.

Unsupervised learning finds structure in unlabelled data, e.g. clustering, dimensionality reduction, and anomaly detection.

Reinforcement learning trains an agent that selects actions to maximise cumulative reward through interaction with an environment; key elements are state, action, reward, policy.

Semi-supervised learning combines a small labelled set with a large unlabelled set; self-supervised learning derives labels from the data itself.

Bias is error from oversimplified assumptions; variance is error from sensitivity to training data. High bias underfits, high variance overfits; total error = bias^2 + variance + irreducible noise.

The bias-variance trade-off means increasing model complexity typically lowers bias and raises variance; the goal is the complexity that minimises generalisation error.

Overfitting is strong training performance with weak test performance; remedies include more data, regularisation, simpler models, and cross-validation.

The ML workflow: problem framing, data collection, data preparation, model selection, training, evaluation, and deployment with monitoring.

Python is the dominant ML language; core libraries are NumPy for arrays, pandas for tabular data, scikit-learn for classical algorithms, and TensorFlow or PyTorch for deep learning.

NumPy ndarrays support vectorised operations, broadcasting, and element-wise arithmetic, which replace slow Python loops.

scikit-learn's API is fit(X, y) to train, predict(X) to infer, and transform(X) for preprocessing; all estimators share this consistent interface.

Train/test split (commonly 70/30 or 80/20) estimates generalisation; k-fold cross-validation rotates the validation fold k times and averages the scores.

Data leakage occurs when information from the test set influences training, e.g. fitting a scaler on all data before splitting; fit transformers on training data only.

Pipelines chain transformers and an estimator so preprocessing is fitted inside each cross-validation fold, preventing leakage.

Feature engineering creates or transforms inputs: one-hot encoding for categorical variables, scaling for distance-based models, imputation for missing values.

Hyperparameters are set before training (e.g. k in k-NN, tree depth); grid search and random search tune them using cross-validation.

Reproducibility requires fixing random seeds, versioning data and code, and recording library versions.

Linear regression fits y = wX + b by minimising mean squared error; coefficients are estimated by ordinary least squares or gradient descent.

Logistic regression applies the sigmoid to a linear combination to output probabilities for binary classification; it is trained by minimising log loss.

k-Nearest Neighbours classifies a point by the majority label of its k nearest neighbours; it is non-parametric, lazy, and sensitive to feature scaling.

Decision trees split data on features that maximise impurity reduction (Gini or entropy); deep trees overfit, so pruning or depth limits are used.

Random forests average many decorrelated trees trained on bootstrap samples with random feature subsets, reducing variance versus a single tree.

Gradient boosting builds trees sequentially, each fitting the residual errors of the previous ensemble; it usually achieves high accuracy but needs careful tuning.

Support Vector Machines find the maximum-margin hyperplane; the kernel trick (linear, polynomial, RBF) handles non-linear boundaries.

k-means partitions data into k clusters by iteratively assigning points to the nearest centroid and recomputing centroids; k must be chosen, e.g. by the elbow method.

Principal Component Analysis projects data onto orthogonal components that maximise variance; it is used for dimensionality reduction and visualisation.

Classification metrics: accuracy, precision = TP/(TP+FP), recall = TP/(TP+FN), F1 = harmonic mean of precision and recall.

The confusion matrix tabulates true/false positives and negatives; accuracy is misleading on imbalanced classes, where precision, recall, and AUC are preferred.

Regression metrics: MAE averages absolute errors, MSE and RMSE penalise large errors more heavily, and R-squared is the proportion of variance explained.

ROC curves plot true positive rate against false positive rate; AUC near 1 indicates strong discrimination, 0.5 is random.

Use cases: fraud detection and spam filtering (classification), demand forecasting (regression), customer segmentation (clustering), recommender systems (ranking).

Deployment requires monitoring for data drift and concept drift, where input distributions or input-output relationships change after training.

Explainability techniques such as feature importance, SHAP, and LIME help stakeholders understand model decisions in regulated domains.

Ethical and legal concerns include algorithmic bias, fairness, privacy under GDPR, consent, and the right to explanation for automated decisions.

Model selection should match the business objective: a false negative in medical screening often costs more than a false positive, favouring high recall.