ICML study plan
ICML · Give each week a clear focus.
8 weeks · About 7 hours per week
Mathematics and Statistics Foundations
Derive the bias-variance decomposition from scratch and write it out without notes
Compute gradients and Hessians for common loss functions (MSE, logistic, softmax cross-entropy) by hand
Solve 5 linear-algebra problems on eigenvalues, SVD, and positive definiteness
Explain the difference between MLE and MAP with a worked example
Your checkpoint
A one-page handwritten-style reference sheet of key derivations (bias-variance, gradients, MLE vs MAP) scanned and self-graded against a checklist
Mathematics and Statistics Foundations & Supervised Learning
Derive the closed-form solution for ridge regression and connect it to the normal equations
Implement logistic regression from scratch with gradient descent and compare to a library implementation
Prove or disprove convexity for three given loss functions
Work through 10 probability questions on conditional expectation, Bayes' rule, and concentration inequalities
Your checkpoint
A notebook implementing logistic and ridge regression from scratch, with a written comparison of test accuracy and convergence to library baselines
Supervised Learning
Implement and tune at least three model families (regularized linear, gradient-boosted trees, kernel or SVM-based) on one dataset
Explain how regularization strength interacts with bias and variance using learning curves
Derive the dual formulation of the soft-margin SVM and summarize the role of support vectors
Diagnose underfitting vs overfitting from learning curves and state the corrective action for each
Your checkpoint
A model comparison report on one dataset with learning curves, hyperparameter tables, and a written diagnosis of each model's failure mode
Model Evaluation and Validation
Design a nested cross-validation protocol and justify each design choice in writing
Compute and interpret precision, recall, F1, ROC-AUC, and PR-AUC on an imbalanced dataset
Explain when accuracy is misleading and demonstrate the failure with a concrete example
Identify and correct three leakage scenarios in provided pipeline descriptions
Your checkpoint
A validation protocol document plus a notebook showing metrics on an imbalanced dataset with a written justification of the chosen primary metric
Data Preparation and Feature Engineering
Build a reproducible preprocessing pipeline handling missing values, categorical encoding, and scaling
Demonstrate and then fix target leakage introduced by fitting transforms before splitting
Engineer at least five features and quantify each one's marginal contribution to validation performance
Compare three imputation strategies and report the performance impact of each
Your checkpoint
A reusable preprocessing module with unit tests plus a short ablation table showing the effect of each feature and imputation choice
Deep Learning and Neural Networks
Implement a small neural network from scratch (forward and backward pass) and verify gradients numerically
Explain and demonstrate the effect of learning rate, batch size, and initialization on training stability
Apply at least two regularization techniques (dropout, weight decay, early stopping) and measure their effect
Diagnose overfitting and vanishing gradients from training curves and state a remedy for each
Your checkpoint
A from-scratch neural network notebook with a numerical gradient check and a training-curve experiment log covering at least four hyperparameter settings
Unsupervised Learning
Implement k-means and compare its results against a density-based method on the same data
Apply PCA and explain the variance tradeoff as the number of components changes
Select the number of clusters using at least two quantitative criteria and justify the choice
Interpret cluster assignments and state at least one failure mode of each clustering method used
Your checkpoint
A clustering study with a component/cluster selection plot, a comparison table across methods, and written interpretation of the resulting clusters
ML Engineering, Deployment, and MLOps & Responsible AI, Ethics, and Governance
Package a trained model behind an inference endpoint with input validation and versioned artifacts
Write a monitoring plan covering data drift, performance decay, and rollback triggers
Identify bias and fairness risks in a provided model card and propose concrete mitigations
Draft the governance artifacts (model card, intended use, limitations) for your week 3 model
Your checkpoint
A deployed model endpoint with a README, a monitoring plan, and a completed model card covering intended use, limitations, and fairness considerations
Milestones
Week 2: Can derive the bias-variance decomposition and the ridge closed-form solution from memory, and has a working from-scratch logistic regression that matches library accuracy within 1 point
Week 4: Has a documented nested cross-validation protocol and can correctly identify all three leakage scenarios without hints
Week 6: From-scratch neural network passes numerical gradient check and training curves show a measurable improvement from at least two regularization techniques
Week 8: Model is deployed behind an endpoint with a monitoring plan and completed model card, and a full mock exam across all eight domains scores at least 75% with every domain above 60%