MLARM flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Residual Risk

The risk remaining after controls are applied. Computed as inherent risk reduced by control effectiveness, and compared against the risk appetite to decide whether further mitigation or approval escalation is required.

Bias-Variance Tradeoff

Expected squared prediction error decomposes into squared bias, variance, and irreducible noise. Increasing model complexity typically lowers bias but raises variance, so model selection balances the two.

Data Leakage

Information from the target or from the test period enters training features, inflating offline performance. Common causes are using post-outcome variables or scaling before splitting the data.

Precision-Recall Tradeoff

Lowering a classification threshold raises recall and lowers precision. The operating threshold should be chosen from the business cost of false positives versus false negatives, not from default settings.

Three Lines Model

Governance structure where the business owns risk, an independent risk and compliance function provides oversight and challenge, and internal audit gives independent assurance to the board.

SR 11-7

US supervisory guidance on model risk management for banking organizations. It sets expectations for model development, validation, governance, inventory, and ongoing monitoring of models in use.

EU AI Act High-Risk

Classification for AI systems used in areas such as critical infrastructure, employment, education, essential services, and law enforcement. It triggers obligations for risk management, data governance, documentation, logging, and human oversight.

Model Inventory

A maintained register of all models recording identifier, owner, purpose, risk tier, dependencies, validation status, and review dates. It is the foundation for scoping validation and monitoring work.

NIST AI RMF

A voluntary framework organized into Govern, Map, Measure, and Manage functions. Govern is cross-cutting; the other functions structure identification, assessment, and treatment of AI risks.

Differential Privacy

A guarantee that adding or removing one individual's record changes output distributions only slightly, controlled by a privacy budget epsilon. Smaller epsilon means stronger privacy and more noise.

k-Anonymity

A release property where each record is indistinguishable from at least k-1 others on quasi-identifiers. It limits re-identification but does not protect against inference from sensitive attribute homogeneity.

Data Poisoning

An attack that corrupts training data to degrade model performance or implant a backdoor trigger. Defenses include data provenance checks, outlier screening, and training-set auditing.

Membership Inference

A privacy attack that determines whether a specific record was in the training set by exploiting model confidence differences. Mitigations include regularization, output perturbation, and differential privacy.

Train-Serve Skew

A mismatch between feature computation in training and in production, causing live performance to fall below offline estimates. Detection uses parity tests and distribution comparison of recomputed features.

Model Card

A short standardized document describing a model's intended use, training data, evaluation results by segment, limitations, and ethical considerations. It supports transparency for users and reviewers.

Effective Challenge

Independent validation that critically tests a model's conceptual soundness, data, and outcomes, performed by staff separate from developers with authority to require remediation or reject the model.

Backtesting

Comparing model predictions against realized outcomes over historical periods. It tests discriminatory power and calibration, and is a core component of outcomes analysis in validation.

Champion-Challenger

A validation and deployment pattern where a challenger model runs alongside the incumbent champion on the same data or traffic, and results are compared before any replacement decision.

Concept Drift

A change in the relationship between inputs and target, so P(Y|X) shifts and the learned mapping decays. It requires retraining rather than only input monitoring.

Covariate Shift

A change in the input distribution P(X) while the conditional relationship P(Y|X) stays stable. Performance may degrade where the new inputs fall outside the training region.

Population Stability Index

A drift metric comparing binned distributions between a baseline and current sample. Common interpretation: below 0.1 stable, 0.1 to 0.25 moderate shift, above 0.25 significant shift.

Shadow Deployment

Running a new model in parallel with the production model on live inputs without using its outputs for decisions. It measures real-world behavior and latency before a controlled rollout.

SHAP Values

Shapley-value attributions that assign each feature a contribution to a single prediction, summing to the prediction minus the baseline. They are additive but costly and sensitive to correlated features.

Equalized Odds

A fairness criterion requiring equal true positive rates and equal false positive rates across protected groups. It constrains error rates rather than selection rates.

Demographic Parity

A fairness criterion requiring the same positive prediction rate across protected groups. It ignores base rates and can conflict with predictive parity and equalized odds.

Proxy Variable

A feature correlated with a protected attribute, such as postal code or surname. It can transmit discriminatory signal even when the protected attribute itself is excluded.

Counterfactual Explanation

An explanation that identifies the smallest change to input features that would alter a model's decision. It is intuitive for adverse-action contexts but may suggest infeasible changes.

Model Decommissioning

The controlled retirement of a model: revoking access, archiving code, data, and validation records for the retention period, updating the inventory, and migrating dependent processes.

Fourth-Party Risk

Risk introduced by a vendor's own subcontractors, such as cloud subprocessors hosting model data. It requires mapping the supply chain and confirming equivalent contractual and security controls.

Material Model Change

A change substantial enough to require revalidation, such as new data sources, altered methodology, or expanded use. Minor changes follow a documented lighter review path.

Prompt Injection

An attack where untrusted input manipulates a language model into ignoring instructions or leaking data. Defenses include input isolation, least-privilege tool access, and output filtering.

Human Oversight

A control requiring meaningful human review or intervention capability over AI decisions. Its form, per-decision approval or monitoring with override, depends on impact and reversibility.