INFORMS CAP flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Decision-relevant problem statement

A framing that names the decision, decision maker, available actions, objective, and constraints, so that modeling effort maps to a real choice rather than a vague request.

Counterfactual baseline

The outcome the organization would achieve without the model, used as the reference against which any claimed model value is measured.

Value of a model

The improvement in decision outcomes attributable to the model minus the cost of building and operating it, expressed in the decision maker's units.

Stakeholder role map

An explicit list of sponsor, decision owner, data owner, and affected parties; a missing role predicts later deployment or adoption failure.

Falsifiable success criteria

A metric, baseline, target, evaluation window, and stop threshold agreed before work begins, allowing the project to be judged objectively.

CRISP-DM phases

Business understanding, data understanding, data preparation, modeling, evaluation, and deployment; the process is iterative rather than strictly linear.

Causal versus predictive

Predictive models estimate outcomes from observed features; causal questions require design-based identification such as randomization or instrumental variables.

Cross-validation variants

k-fold for independent data, stratified for imbalance, time-series split for temporal order, and group k-fold when entities repeat across folds.

Regularization L1 versus L2

L1 penalizes absolute coefficients and can zero them out for selection; L2 penalizes squares and shrinks coefficients smoothly without selection.

Optimization formulation

An explicit statement of decision variables, objective function, and constraints, checked for feasibility and boundedness before solver results are trusted.

Missingness mechanisms

MCAR means missing independently of values, MAR means predictable from observed data, and MNAR means depending on the unobserved value itself.

Data leakage

Use of information unavailable at decision time, or derived from the target, as a feature; it inflates validation scores and fails in production.

Class imbalance reporting

When positives are rare, report precision, recall, and PR-AUC rather than accuracy, and tune the decision threshold to the cost matrix.

Data lineage record

Documentation of source, extraction date, transformations, and owner for each field, enabling reproduction and audit of any model input.

Sampling representativeness

The evaluation sample must mirror the deployment population; otherwise measured performance will not transfer to live decisions.

Bias-variance decomposition

Expected prediction error equals squared bias plus variance plus irreducible noise; added model capacity typically lowers bias and raises variance.

Overfitting signal

Training error substantially below validation error; remedies include more data, regularization, a simpler model, or early stopping.

Probability calibration

Agreement between predicted probabilities and observed frequencies, checked with reliability curves and corrected by Platt scaling or isotonic regression.

Threshold as business decision

The operating point should be chosen from the relative cost of false positives and false negatives, not from an arbitrary default such as 0.5.

Feature importance caveat

Importance measures dependence within a fitted model and is not causal evidence; correlated features can distort permutation and split-based rankings.

Reproducible model pipeline

Fixed seeds, pinned library versions, versioned data snapshots, and a scripted path from raw data to final artifact that can be rerun exactly.

Shadow deployment

Running a model alongside current practice without acting on its output, to validate inputs, latency, and score distribution before decisions change.

Training-serving skew

Divergence between feature transformations in training and in production serving; a leading cause of silent model failure after release.

Canary release with rollback

A staged rollout to a small share of traffic with predefined error, latency, and drift triggers and an authorized owner who can revert.

Deployment monitoring triad

Tracking input distribution drift, output score distribution and calibration, and realized business outcomes, each with alert thresholds and owners.

Joint artifact versioning

Model artifact, code, configuration, and training data snapshot are versioned together so any historical prediction can be reproduced.

Drift types

Covariate drift changes input distribution, label drift changes target distribution, and concept drift changes the input-output relationship; responses differ.

Retraining trigger

A predefined condition for refitting a model: scheduled cadence, performance below threshold, drift beyond a control limit, or upstream data change.

Independent model validation

Review of a model by someone other than its developer, with documented findings and sign-off, required for high-risk production models.

Model inventory

A register recording purpose, owner, version, risk tier, data sources, validation results, and approval date for every production model.

Fairness metric incompatibility

Demographic parity, equalized odds, and predictive parity generally cannot all hold at once, so the chosen criterion must be justified by context.

Honest reporting obligation

Reporting negative findings, uncertainty, and all relevant runs, and never presenting correlational predictions as causal claims to decision makers.