MS MLO flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Data drift

A change over time in the distribution of model input features compared with the training distribution. Detected by comparing feature statistics such as histograms or PSI between baseline and current scoring data.

Concept drift

A change in the underlying relationship between inputs and target, so the same input now maps to a different outcome. Input drift checks alone may miss it; labeled outcomes or delayed evaluation are needed.

Population stability index

A numeric drift score comparing the distribution of a variable across two samples using binned proportions. Roughly below 0.1 indicates stability, above 0.2 meaningful shift, above 0.25 retraining consideration.

Managed online endpoint

An Azure ML resource exposing a real-time HTTPS scoring URL backed by one or more deployments. Supports traffic splitting by percentage, autoscaling, key or Azure AD token authentication, and private networking.

Batch endpoint

An Azure ML endpoint that runs scoring jobs over a registered data asset rather than per-request calls. Suitable for large files and long-running inference, writing scored output to a configured datastore.

Canary deployment

A rollout pattern that routes a small percentage of live traffic to a new deployment while the rest stays on the stable version. Traffic increases gradually as health metrics remain within thresholds.

Blue-green deployment

A rollout pattern with two parallel environments where traffic switches fully from the old version to the new one. Rollback is immediate because the previous environment remains deployed and addressable.

Scoring script init function

The entry point in an Azure ML scoring script that runs once when the deployment starts. It loads the model and any heavy dependencies so per-request scoring stays fast.

Scoring script run function

The per-request entry point in an Azure ML scoring script. It receives request payload data, applies preprocessing and prediction, and must return a JSON-serializable result.

Hyperparameter tuning sampling

The strategy selecting parameter values per trial: grid enumerates a fixed set, random samples independently, and Bayesian uses results of prior trials to choose promising values when trials are costly.

Early termination policy

A Hyperdrive setting that stops poorly performing trials before completion. Bandit compares against the best run with a slack factor; median stopping compares against the running median of the primary metric.

Stratified k-fold cross-validation

A cross-validation scheme that preserves the class proportion of the full dataset in each fold. Preferred over plain k-fold when classes are imbalanced, because each fold remains representative.

Group k-fold

Cross-validation that keeps all records belonging to the same entity, such as a patient or user, within one fold. Prevents leakage when multiple rows per entity exist in the dataset.

PR-AUC versus ROC-AUC

PR-AUC plots precision against recall and is preferred when positives are rare because it focuses on the minority class. ROC-AUC can look strong despite poor minority-class performance under heavy imbalance.

Target leakage

A feature that encodes information unavailable at prediction time, such as a post-outcome field or statistics computed over the full dataset. Produces optimistic validation scores that collapse in production.

Feature store

A system providing one definition of each feature reused across training and inference. An offline store serves batch training; an online store serves low-latency feature lookup at scoring time.

Point-in-time join

A feature join that retrieves the feature value as it existed at the event timestamp rather than the latest value. Prevents look-ahead bias in time-dependent training data.

MLTable data asset

An Azure ML data asset type that references tabular data with a declared schema and lazy loading. Enables schema validation and typed columns when defining training inputs in jobs.

Imputation inside folds

Computing missing-value replacement statistics only from the training fold, then applying them to validation data. Computing statistics on the full dataset leaks validation information into training.

Model card

A governance document describing a model's intended use, limitations, training data, evaluation metrics including subgroup results, and ethical considerations. Supports review and audit before production approval.

Demographic parity

A fairness criterion requiring the positive prediction rate to be similar across protected groups. It ignores actual outcomes, so it can conflict with equalized odds when base rates differ between groups.

Equalized odds

A fairness criterion requiring true positive rates and false positive rates to be similar across protected groups. It accounts for actual outcomes, unlike demographic parity, which only compares positive rates.

SHAP local explanation

A per-prediction attribution method that assigns each feature a contribution relative to a baseline. Used to justify individual decisions and to debug unexpected model behavior for one record.

Permutation feature importance

A global explanation technique that shuffles one feature at a time and measures the resulting drop in model performance. Model-agnostic but can mislead when features are strongly correlated.

Differential privacy

A privacy technique adding calibrated noise so that any single individual's data has bounded influence on outputs. Provides a formal guarantee traded against model accuracy through the privacy budget.

Managed identity for workspace

An Azure AD identity assigned to the Azure ML workspace or compute so jobs access storage and Key Vault without stored credentials. Preferred over secrets in code or configuration files.

Azure ML component

A reusable, versioned pipeline step defined in YAML with typed inputs and outputs, code, and environment. Components compose into pipelines and can be shared across experiments and teams.

Model registration gate

A CI/CD condition that registers or promotes a model only when it beats the current production model on the primary metric and passes fairness and drift checks on a holdout set.

Infrastructure as code for ML

Defining workspaces, compute, datastores, and endpoints in declarative templates such as Bicep or Terraform. Enables repeatable, reviewable environment creation and drift detection across dev, test, and production.

Federated credential in CI

An OIDC trust between a CI provider and Azure AD that lets pipeline runs authenticate without stored secrets. Replaces long-lived service principal secrets used to reach Azure ML.

Compute cluster versus instance

A compute instance is a single managed VM for interactive development such as notebooks. A compute cluster autoscales across nodes for training and batch jobs and scales to zero when idle.

Batch versus online scoring

Online scoring answers individual requests with low latency through an endpoint. Batch scoring processes large stored datasets on a schedule and writes predictions to a datastore for downstream consumption.