Model Interpretation and Explainability
What this domain tests
Model Interpretation and Explainability tests whether you can explain global behavior, local predictions, and the limits of each explanation method in H2O-3 and Driverless AI.
Core rules
- Global methods summarize the model over a population: variable importance, partial dependence, and global SHAP summaries. Local methods explain one row: SHAP contributions, LIME, and reason codes.
- H2O-3 provides
h2o.varimp(),h2o.partialPlot(),h2o.predict_contributions()with SHAP output for supported models, andh2o.explain()for a combined view. - Variable importance is model-specific. Split-based importance can favor high-cardinality or correlated features; gain and cover tell different stories. Importance is not causality.
- Partial dependence shows the average predicted response as a feature changes, holding others fixed. It can be misleading when features are strongly correlated, because the hypothetical combinations may not exist.
- Driverless AI MLI includes variable importance, partial dependence, SHAP, LIME, surrogate models, sensitivity analysis, and model documentation. Use the documentation view for governance, not just the headline chart.
- For local explanations, check the baseline and the scale. SHAP values are additive contributions relative to a baseline; they are not direct probabilities.
Worked example
A loan default model ranks income as the most important feature, but income is correlated with loan_amount. A partial dependence plot for income alone shows a smooth decline in default risk, but it averages over unrealistic loan amounts. Plot both features together, then use a SHAP waterfall for a declined applicant to see whether the decision was driven by low income, high loan amount, or both. The reasoning is that correlated features distort single-feature explanations.
Traps
- Treating variable importance as a causal effect.
- Comparing importance scores across models trained with different encodings.
- Reading a partial dependence plot as if it were a causal dose-response curve.
- Forgetting that one-hot encoding splits importance across dummy columns.
- Presenting a local SHAP explanation without stating its baseline.