MS MLO study plan
MS MLO · Give each week a clear focus.
8 weeks · About 7 hours per week
Design and implement ML solutions
Map the end-to-end ML lifecycle into a one-page diagram and label where each of the seven exam domains touches it.
Decompose a business problem into a formal ML problem statement: define the target, label source, prediction unit, and success metric.
Compare at least four algorithm families (linear, tree ensemble, neural, clustering) on bias-variance behavior and data requirements, and write a selection checklist.
Design a training pipeline for a tabular use case that names data splits, leakage risks, and baseline model choice.
Your checkpoint
A two-page design brief for one tabular ML solution containing the lifecycle diagram, problem statement, algorithm selection checklist, and pipeline sketch.
Design and implement ML solutions & Data engineering and feature management
Implement a reproducible data preparation script that ingests raw data, validates schema, and writes versioned feature tables.
Detect and handle missing values, outliers, and class imbalance using at least three documented strategies each with a rationale.
Build a feature store pattern in miniature: separate offline training features from online serving features and reconcile them.
Measure training-serving skew by comparing feature statistics from batch and simulated real-time paths.
Your checkpoint
A runnable repository with a data validation script, feature engineering notebook, and a short report showing batch vs. online feature statistics side by side.
Data engineering and feature management & Model training, tuning, and experimentation
Engineer at least ten features across numeric, categorical, and time-based types and document encoding choices.
Set up an experiment tracker and log five training runs with hyperparameters, metrics, and artifacts.
Perform hyperparameter tuning with a documented search space and compare random search against Bayesian optimization.
Diagnose overfitting and underfitting from learning curves and apply at least two regularization techniques.
Your checkpoint
An experiment log table of five or more tracked runs plus a one-page tuning report naming the best configuration and the evidence supporting it.
Model training, tuning, and experimentation
Design a cross-validation scheme appropriate to the data's temporal or grouped structure and justify it.
Select evaluation metrics that match the business cost of errors, including precision-recall tradeoffs and calibration.
Run a fair model comparison across three algorithms using identical splits and seeds.
Write a model card draft covering intended use, training data, metrics, and known limitations.
Your checkpoint
A model selection report with a comparison table of three algorithms, chosen metric rationale, and a completed model card draft.
Design and implement ML solutions & Model deployment and serving
Package a trained model into a containerized inference service with a health endpoint and input schema validation.
Compare batch, real-time, and streaming serving patterns and choose one for a stated latency and throughput requirement.
Implement a canary or blue-green rollout plan including rollback criteria.
Load-test the service and record latency percentiles under increasing concurrency.
Your checkpoint
A deployed containerized model endpoint with a README documenting the serving pattern choice, rollout plan, and measured p50 and p95 latency.
Monitoring, observability, and maintenance
Instrument the deployed model to log inputs, predictions, and latency to a monitoring store.
Define data drift and concept drift detectors with thresholds and alert routing.
Build a dashboard showing prediction distribution, feature drift, and error rate over time.
Write a runbook for responding to a drift alert, including retraining triggers.
Your checkpoint
A monitoring dashboard screenshot or export plus a runbook that specifies alert thresholds and the retraining decision path.
Responsible AI, security, and governance
Produce a threat model for the ML system covering data poisoning, model theft, and adversarial input risks.
Apply access control and secrets management to the training and serving pipeline, documenting each control.
Evaluate the model for fairness across at least two sensitive subgroups and document mitigation options.
Draft a governance checklist covering data lineage, consent, retention, and audit logging.
Your checkpoint
A responsible AI and security dossier containing the threat model, fairness evaluation table, and governance checklist.
CI/CD, automation, and infrastructure for MLOps & Design and implement ML solutions
Automate the full pipeline from data validation through training to deployment in a CI/CD workflow.
Add automated tests for data schema, model performance regression, and endpoint contract.
Implement infrastructure as code for the serving environment and reproduce it from scratch.
Complete two timed practice sets across all domains and log every missed concept with a corrective action.
Your checkpoint
A working CI/CD pipeline file that runs tests and gates deployment on a performance threshold, plus a remediation log of missed practice concepts.
Milestones
Week 2: Reproducible data pipeline runs end to end from a clean checkout and produces identical feature statistics on rerun.
Week 4: Three algorithms compared on identical splits with a documented metric rationale; model card draft reviewed against a checklist.
Week 6: Deployed endpoint is monitored with drift and latency dashboards, and a drift alert has been triggered and resolved using the runbook.
Week 8: CI/CD pipeline blocks a deliberately degraded model and passes a healthy one; practice scores logged with remediation actions for every missed concept.