Modeling: Algorithm Choice, Training, and Evaluation
Frame, select, tune, evaluate
Modeling is the largest domain. Expect questions that give a data shape and business goal and ask for the algorithm or metric.
Built-in algorithms. XGBoost for tabular classification/regression and the usual correct answer for mixed-type tabular data; Linear Learner for high-dimensional sparse linear baselines; Factorization Machines for sparse interaction data such as recommenders; k-NN for small, low-dimensional similarity search; k-means for clustering (choose k by elbow or silhouette); Random Cut Forest for anomaly detection; DeepAR for forecasting many related time series; BlazingText for text classification and word vectors; Object Detection, Image Classification, and Semantic Segmentation for vision; LDA for topic modeling.
Training controls. Regularization (L1/L2, max_depth, min_child_weight), early stopping on a validation metric, learning-rate schedules, and distributed training modes. Automatic Model Tuning runs hyperparameter search jobs (Bayesian by default, with random, grid, and Hyperband strategies), which requires fixed objective metrics and sensible ranges - it cannot rescue bad features.
Metrics. Regression: RMSE (penalizes large errors), MAE (robust), R². Classification: log loss and AUC for ranking and probability quality, F1/precision/recall for imbalance, accuracy only when classes are balanced. Multiclass tuning often needs a single scalar objective; use a macro- or micro-averaged metric.
Traps. Never tune or select on the test set. AUC is threshold-independent, so it can look strong while the deployed threshold is bad. Confusing an objective function with an evaluation metric in the XGBoost hyperparameters breaks training.
Worked example. Tabular binary classification with ~5% positives, missing values, and mixed categorical/numeric columns: use XGBoost (handles missing values natively), set the objective to binary:logistic, the eval metric to auc, add early stopping, and tune max_depth, eta, subsample, and min_child_weight rather than guessing.