H2O.ai Certified Machine Learning flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Bias-variance decomposition

Expected squared prediction error splits into squared bias, variance, and irreducible noise. High bias underfits; high variance shows a wide gap between training and validation error.

Precision-recall tradeoff

Lowering the classification threshold raises recall and lowers precision. The F1-optimal threshold is read from the threshold table rather than assumed to be 0.5.

ROC-AUC interpretation

ROC-AUC is the probability that a randomly chosen positive scores above a randomly chosen negative. It ignores prevalence, so rare-event problems need PR-AUC instead.

Stratified k-fold

Cross-validation that preserves class proportions in every fold. It stabilizes the error estimate when one class is rare, and grouped folds are needed when rows share an entity.

Log loss formula

Log loss averages the negative log probability assigned to the true class. Confident wrong predictions are penalized heavily, and it is H2O-3's default classification stopping metric.

H2O-3 cluster model

A JVM cluster exposing a REST API over distributed in-memory column-compressed frames. Python, R, and Flow clients are thin wrappers that send REST calls to the cluster.

MOJO artifact

A model object, java optimized, exported as a zip that scores with the h2o-genmodel runtime. It is version-tolerant and needs no training cluster at serving time.

POJO versus MOJO

A POJO is a compiled Java class tied to the H2O build version. A MOJO is a portable zip that tolerates version changes and is preferred for long-lived production deployment.

Driverless AI experiment

An automated run that evolves features, tunes models, and outputs a portable scoring pipeline. It is a separate product from H2O-3 and freezes engineering into the artifact.

Cluster memory sizing

Aggregate cluster memory must exceed the largest frame plus model working memory. Deep learning and stacked ensembles need more headroom than GLM or single tree models.

H2O Wave role

Wave is the application and dashboard layer built over H2O models. It renders interfaces and calls scoring endpoints; it is not itself the model-serving runtime.

Target leakage detection

A feature with near-perfect univariate association to the label usually leaks the outcome. Cross-validation will not expose it, so drop such features before training.

High-cardinality categoricals

Categorical columns with thousands of levels inflate memory and invite overfitting. Mitigations are target encoding, frequency encoding, or grouping rare levels together.

Missing value handling

H2O-3 imputes numeric missing values by default and treats categorical missingness as its own level. fillna and h2o.impute make the chosen treatment explicit and reproducible.

Variance inflation factor

VIF quantifies how much a predictor's variance is inflated by correlation with others. Values above roughly ten signal collinearity; regularized GLM or dropping a feature resolves it.

Feature scaling relevance

Scaling affects distance- and gradient-based models such as regularized GLM, k-means, and deep learning. Tree splits are invariant to monotone rescaling, so scaling is unnecessary there.

H2O AutoML run order

AutoML trains baseline models, then random hyperparameter grids, then stacked ensembles. max_models or max_runtime_secs bounds the total search effort and runtime.

Early stopping requirements

H2O-3 GBM and XGBoost early stopping needs a validation frame plus stopping_rounds, stopping_metric, and stopping_tolerance. Without a validation frame it never triggers.

GBM core hyperparameters

ntrees, max_depth, min_rows, learn_rate, sample_rate, and col_sample_rate control capacity and regularization. Lower learn rates need more trees but usually generalize better.

Stacked ensemble metalearner

Stacking uses cross-validated base-model predictions as level-one features, with a GLM metalearner by default. Ensemble quality depends on consistent fold assignment across base models.

H2O grid search

H2OGridSearch evaluates a hyperparameter map exhaustively. Random search is preferred when the space is large and each model evaluation is expensive.

GLM lambda search

lambda_search walks the regularization path instead of evaluating a full grid. It pairs with alpha to mix L1 and L2 penalties efficiently during tuning.

Split-based variable importance

Tree importance sums weighted split gains, which biases toward high-cardinality features. Corroborate it with permutation importance before drawing conclusions about feature value.

Permutation importance

Shuffle one feature's values and measure the metric drop. It is model-agnostic and reveals predictive reliance, but correlated features can mask each other's contribution.

Partial dependence plots

Partial dependence plots average the model response as one feature varies. They assume feature independence, so they mislead when predictors are strongly correlated in the training data.

SHAP additivity

SHAP attributions for one row sum to the prediction minus the baseline expectation. TreeSHAP computes exact values for tree ensembles in polynomial time.

H2O reason codes

Per-row feature contributions explaining an individual prediction. H2O-3 supports them for GLM, GBM, DRF, XGBoost, and stacked ensembles as the local counterpart to global importance.

Residual structure diagnosis

Plotting residuals against features and predictions reveals patterns. Structure indicates an unmodeled interaction, a missing transform, or heteroscedasticity that a metric alone hides.

Scoring parity check

Score the same holdout rows in the training cluster and through the production path. Predictions must match within a tight tolerance before traffic is cut over.

Model drift monitoring

Compare live feature and prediction distributions against training baselines using PSI or KS statistics. Threshold breaches trigger retraining or rollback of the deployed version.

Model registry contents

A registry stores the artifact, training data fingerprint, hyperparameters, metrics, and version. Stage promotion is an auditable state change that supports rollback.

MOJO serving concurrency

A MOJO is thread-safe; each row scores single-threaded in microseconds to low milliseconds. Throughput scales by adding concurrent requests rather than internal parallelism.