Alteryx Machine Learning cheat sheet

Supervised learning requires a labeled target column; unsupervised learning (clustering, association) has no target and is used for segmentation and pattern discovery.

Classification targets are categorical (binary or multiclass); regression targets are continuous numeric; the target type determines the loss function and evaluation metrics.

Binary classification metrics derive from the confusion matrix: TP, FP, TN, FN; accuracy = (TP+TN)/(TP+TN+FP+FN).

Precision = TP/(TP+FP); Recall = TP/(TP+FN); F1 = 2·(Precision·Recall)/(Precision+Recall), the harmonic mean.

For regression, MAE averages absolute errors, RMSE averages squared errors then takes the square root, and R² = 1 − SS_res/SS_tot.

Class imbalance skews accuracy; use precision, recall, F1, or AUC instead when the minority class is the class of interest.

A well-framed problem states the prediction unit (row grain), the target, the prediction time horizon, and the decision the prediction supports.

Data leakage occurs when features encode information unavailable at prediction time (e.g., a post-outcome field); it inflates validation scores.

Train/validation/test splits separate model fitting, hyperparameter selection, and final unbiased evaluation; the test set is touched once.

Missing numeric values can be imputed with mean, median, or a constant; missing categorical values with mode or an explicit 'Unknown' category.

One-hot encoding converts a categorical field with k levels into k (or k−1) binary indicator columns; high-cardinality fields create sparse, wide data.

Numeric scaling matters for distance- and gradient-based algorithms: standardization gives mean 0 and SD 1; min-max scaling maps values to [0,1].

Date fields should be decomposed into components (year, month, day-of-week, day-of-year) plus derived durations such as days since a reference event.

Feature engineering creates ratios, differences, aggregates, and time-window statistics that expose signal the raw columns do not carry directly.

Outliers can be detected via z-scores (|z| > 3), IQR fences (Q1 − 1.5·IQR, Q3 + 1.5·IQR), or domain thresholds; decide to cap, transform, or drop.

Target leakage checks: for each feature, ask whether its value would be known at the moment the prediction must be made.

Fit imputers, scalers, and encoders on training data only, then apply the learned parameters to validation, test, and scoring data.

Alteryx Machine Learning automates problem setup, feature handling, model training, and selection, then exposes results for review and deployment.

Model families typically compared include linear/logistic regression, decision trees, random forests, gradient boosting, and XGBoost-style ensembles.

Hyperparameters control model capacity: tree depth, number of trees/estimators, learning rate, and minimum samples per leaf are common knobs.

Cross-validation (k-fold) estimates generalization by training on k−1 folds and validating on the held-out fold, repeated k times.

The platform ranks candidate models by a leaderboard metric appropriate to the target type (e.g., AUC/F1 for classification, RMSE/MAE for regression).

Feature importance from tree ensembles ranks predictors by contribution; permutation importance measures the score drop when a feature is shuffled.

A baseline model (e.g., majority class or mean prediction) sets the minimum performance a trained model must beat to be useful.

The target column must be excluded from the feature set; identifier columns with no predictive meaning should also be excluded.

ROC-AUC ranges from 0 to 1; 0.5 equals random ranking and higher values indicate better separation of classes.

A confusion matrix at a chosen threshold shows TP, FP, TN, FN; moving the threshold trades precision against recall.

For regression, residual plots reveal non-linearity and heteroscedasticity; systematic patterns indicate model misspecification.

Overfitting shows as high training performance but materially lower validation performance; underfitting shows as poor performance on both.

Cross-validation variance across folds indicates model stability; large fold-to-fold swings suggest high sensitivity to the data sample.

Compare candidate models on the same validation split and the same metric; a single test-set evaluation is the final unbiased estimate.

SHAP values attribute a prediction to individual features, giving per-row and global explanations of model behavior.

Partial dependence plots show the average marginal effect of one feature on predictions, holding other features at observed values.

Calibration assesses whether predicted probabilities match observed frequencies; a well-calibrated 0.8 score implies ~80% positive outcomes.

A trained model is applied to new data in a scoring step; the scoring input must carry the same feature schema and transformations used at training time.

Alteryx workflows chain tools: data input, preparation, model scoring (e.g., a predictive or model-scoring tool), and output to a database or file.

Batch scoring applies the model to a full dataset on a schedule; real-time scoring applies it per request or row.

Schema drift occurs when input columns, types, or categories change after training; scoring pipelines must validate schema before predicting.

The output of scoring typically includes the predicted value and, for classification, a probability or confidence score per class.

Model artifacts must be versioned; record the training data snapshot, feature list, hyperparameters, and metric results with each deployed model.

Scoring thresholds convert probabilities into class labels; the threshold should be set from the business cost of false positives versus false negatives.

Monitoring after deployment tracks input distribution shifts and prediction drift so retraining can be triggered when performance degrades.

Protected attributes (e.g., race, sex, age) and their proxies should be assessed for disparate impact even when excluded from the feature set.

Fairness metrics include demographic parity (equal positive rates across groups) and equalized odds (equal TPR and FPR across groups); they can conflict.

Sensitive data handling requires minimizing collection, restricting access, and complying with applicable privacy regulations.

Model documentation should record purpose, intended use, training data provenance, features, performance, and known limitations.

Human oversight is required for high-stakes decisions; models should inform, not autonomously determine, consequential outcomes.

Explainability requirements vary by audience: global feature importance for stakeholders, per-row explanations for affected individuals.

Governance includes approval workflows, audit logs, periodic review, and a defined owner accountable for each production model.

Retraining policy should specify triggers (performance drop, data drift, elapsed time) and require re-validation before redeployment.