AZ MLO common mistakes
AZ MLO · Understand the traps and how to avoid them.
Treating the workspace and its attached compute as one thing, then wondering why a notebook fails after the compute instance is stopped or a dataset cannot be found from a different compute target.
Candidates learn the portal by clicking around and conflate the workspace (the top-level Azure resource holding assets) with the compute it launches; they do not build a mental model of which assets are workspace-scoped and which live on a specific compute.
What to do instead
Before answering, state the hierarchy out loud: subscription, resource group, workspace, then datastores, datasets, environments, compute, and registered models. For any scenario, ask 'which compute runs this and does it have identity and network access to the store?' and eliminate options that assume a notebook's local files persist across compute changes.
Choosing a compute target for training by picking the largest or cheapest option rather than matching the workload: using a single-node cluster for a distributed framework, or a CPU cluster for a GPU-only training script.
The exam presents compute choices as plausible-looking SKUs and cluster settings; candidates who have only used one setup default to it instead of reasoning from the framework, data size, and cost constraints in the scenario.
What to do instead
Map workload to target deliberately: single-node CPU for small scikit-learn jobs, GPU nodes for deep learning, low-priority nodes for interruptible experiments, and inference clusters or managed online endpoints for serving. Check the scenario for cost, time, and interruption tolerance keywords and let those drive the choice.
Leaking target information during data preparation, for example scaling or imputing with statistics computed on the full dataset before splitting, or using a random split on time-ordered data.
Candidates focus on getting a model to score well and treat preprocessing as a separate cleanup step, so they apply transforms to the whole table, or they ignore that the scenario describes forecasts or transactions over time.
What to do instead
Put every transform inside the training pipeline so it is fit only on training folds. Ask whether the data has a temporal or grouped structure and, if so, split by time or by group (for example by customer or device) rather than at random. Verify by checking that validation metrics are not implausibly better than a later holdout.
Reporting a single accuracy number on an imbalanced classification problem and declaring the model ready, without checking per-class metrics or the baseline.
Accuracy is the default metric in most tutorials and libraries, and candidates under time pressure skip the step of comparing against a majority-class baseline or inspecting precision and recall for the minority class.
What to do instead
Always compute a naive baseline first, then choose metrics that match the cost of errors in the scenario: recall when missing positives is costly, precision when false alarms are costly, and PR curves or AUC for ranking. State the threshold explicitly and show the confusion matrix reasoning rather than one aggregate figure.
Deploying a model to an endpoint without registering it or without pinning the environment and scoring script, so the deployment cannot be reproduced or rolled back.
Candidates treat deployment as a single publish action and skip the asset-management steps that the platform expects; they also assume the local environment will be reconstructed automatically.
What to do instead
Follow the sequence: register the model with a version, define or reference a versioned environment with pinned dependencies, supply an inference script with init and run functions, then deploy to the endpoint. Confirm that the model version and environment version are both recorded so a previous deployment can be restored.
Picking a deployment target by habit (always a real-time endpoint) when the scenario describes batch scoring on a schedule, large volumes, or long-running inference.
Real-time managed online endpoints get the most attention in study material, so candidates default to them and overlook batch endpoints, which suit large files and scheduled jobs without a persistent always-on cost.
What to do instead
Read the scenario for cadence and volume: scheduled bulk scoring over stored data points to a batch endpoint; low-latency interactive requests point to a managed online endpoint; streaming or event-driven scoring points to a pipeline triggered by events. Justify the choice with latency, volume, and cost language from the prompt.
Confusing data drift with concept drift, or monitoring only infrastructure signals such as CPU and latency while ignoring model quality and input distribution changes.
Operational monitoring is familiar from general cloud work, so candidates apply the same dashboards and stop there; the distinction between a changed input distribution and a changed input-to-output relationship is subtle and easy to blur.
What to do instead
Separate the three layers: infrastructure health, data drift (input feature distributions shifting versus a baseline), and concept drift (the relationship between features and target changing, detectable only once labels arrive). For each monitoring scenario, name which layer the signal belongs to and what action it should trigger.
Treating an experiment run as finished once metrics look good, without logging parameters, metrics, or artifacts, so runs cannot be compared or reproduced later.
Candidates run notebooks interactively and view results in the cell output; they do not habitually wrap runs in logging calls or track which dataset version and code produced a result.
What to do instead
Adopt a discipline of one run per experiment with parameters logged before training and metrics and artifacts logged after. When a question asks how to compare or reproduce results, look for the option that records parameters, metrics, tags, and the dataset or code version together, and reject options that rely on notebook history.
Treating fairness as a single metric to maximize, or answering responsible-AI questions with a generic 'add more data' response instead of naming the specific assessment step.
Responsible AI guidance is often read as principles rather than process, so candidates cannot map a scenario about disparate error rates to the concrete tooling and analysis steps that address it.
What to do instead
Learn the workflow: identify sensitive features and cohorts, measure performance disparities across cohorts, examine feature and label provenance for bias, mitigate at the data, training, or post-processing stage, and document the trade-off between fairness and overall accuracy. Match the scenario to the stage where the harm originates.
Assuming interpretability is optional or satisfied by any feature-importance chart, without distinguishing global explanations from per-prediction explanations or checking that the explanation is faithful.
Candidates see feature importance in every tutorial and treat it as a universal answer, missing that regulated or appeal-driven scenarios need local, case-level justification and that some model types require specific explainability tooling.
What to do instead
Classify the need: global understanding of model behaviour versus local justification for an individual decision. Choose tooling that matches the model family, verify the explanation against known behaviour on a test case, and prefer inherently interpretable models when the scenario demands an auditable rule set.