H2O.ai Certified Machine Learning study plan
H2O.ai Certified Machine Learning · Give each week a clear focus.
8 weeks · About 6 hours per week
Model Training, Tuning, and AutoML & Machine Learning Fundamentals and Model Evaluation
Set up a local H2O-3 cluster in Python or R and fit a baseline GBM on a tabular dataset end to end
List the hyperparameters that most affect GBM learning rate, tree depth, and sampling, then explain in writing how each changes bias or variance
Run a small grid search and record training time versus validation AUC for at least six configurations
Distinguish training, validation, and test splits and state which metric each split is allowed to inform
Your checkpoint
A notebook and one-page summary table comparing at least six GBM configurations by validation AUC, training time, and overfitting gap.
Model Training, Tuning, and AutoML & Machine Learning Fundamentals and Model Evaluation
Run H2O AutoML with a fixed time budget and inspect the leaderboard, including stacked ensembles
Explain how cross-validation folds, fold assignment, and early stopping interact during AutoML training
Compute and interpret AUC, logloss, precision-recall, and F1 on an imbalanced target, justifying a metric choice
Reproduce the top AutoML model manually with explicit hyperparameters and confirm comparable validation performance
Your checkpoint
An AutoML run report listing the leaderboard, the chosen metric rationale, and a manual reproduction of the best model within a stated tolerance.
Productionization, Scoring, and MLOps & H2O Platform Architecture and Deployment (H2O-3, Driverless AI)
Export a trained H2O-3 model with h2o.download_mojo and score new rows using the MOJO in a standalone Python process
Deploy the model behind a local REST endpoint and send scoring requests with a JSON payload
Map the H2O-3 architecture components (client, H2O cluster, flow, Steam) and describe which process holds model state
Document memory and CPU sizing rules for a cluster given dataset size and model count
Your checkpoint
A working local scoring service with a MOJO-backed endpoint, plus a short architecture diagram and sizing note.
H2O Platform Architecture and Deployment (H2O-3, Driverless AI) & Productionization, Scoring, and MLOps
Install Driverless AI or run a guided experiment and record how it selects features, models, and ensembles under a time budget
Compare Driverless AI experiment settings against equivalent H2O-3 AutoML settings and note where behavior diverges
Describe how to move an experiment artifact into a scoring pipeline and what reproducibility metadata must travel with it
Identify the failure modes of stale models and write a retraining trigger policy based on drift and performance thresholds
Your checkpoint
A deployment decision memo comparing H2O-3 and Driverless AI for one use case, with a retraining trigger policy and reproducibility checklist.
Data Preparation and Feature Engineering
Build a preprocessing pipeline in H2O-3 that handles missing values, categorical encoding, and skewed numeric features, and measure the effect on validation AUC
Detect and treat outliers and leakage-prone features, documenting each decision and its measured impact
Engineer at least five derived features and rank them by contribution to model performance
Use target encoding with cross-validation folds and verify it does not leak target information
Your checkpoint
A feature engineering log with before-and-after validation metrics for each transformation and a leakage audit table.
Model Interpretation and Explainability
Generate variable importance, partial dependence, and SHAP contributions for a trained model and interpret each correctly
Explain the difference between global and local explanations and choose the right method for a stated stakeholder question
Use reason codes on individual predictions and check that explanations are stable across similar rows
Identify when feature importance misleads under correlated features and state a mitigation
Your checkpoint
An explainability report for one model covering global importance, partial dependence, SHAP, and per-row reason codes, with a caveats section.
Productionization, Scoring, and MLOps & Machine Learning Fundamentals and Model Evaluation
Implement batch scoring on a scheduled job and log predictions, model version, and input hash for auditability
Set up model versioning and a rollback procedure, then rehearse a rollback
Define monitoring for input drift, prediction distribution shift, and latency, with concrete alert thresholds
Re-validate a deployed model against held-out data and document the acceptance criteria for promotion
Your checkpoint
A runnable batch scoring script with version tagging, plus a monitoring and rollback runbook with named thresholds.
Machine Learning Fundamentals and Model Evaluation & H2O Platform Architecture and Deployment (H2O-3, Driverless AI) & Data Preparation and Feature Engineering & Model Training, Tuning, and AutoML & Model Interpretation and Explainability & Productionization, Scoring, and MLOps
Complete a timed full-length practice set and score each domain separately to find weak areas
Re-derive the formulas and definitions you missed, including evaluation metrics and regularization concepts
Re-run one end-to-end pipeline from raw data to scored output without notes
Write a one-page cheat sheet covering metric selection, AutoML controls, and deployment steps
Your checkpoint
A scored practice exam with a per-domain error log and a one-page cheat sheet covering all six domains.
Milestones
Week 2: Can run AutoML on a new dataset, justify the leaderboard metric, and reproduce the best model manually without referring to notes.
Week 4: Can explain the H2O-3 versus Driverless AI division of labor and defend a deployment and retraining plan in a five-minute verbal walkthrough.
Week 6: Can produce and correctly interpret variable importance, partial dependence, SHAP, and reason codes for a model, and name their failure modes.
Week 8: Scores at least 80 percent overall on the practice set with no single domain below 70 percent, and completes the no-notes end-to-end pipeline.