Trees, ensembles, complexity, and explanation
A decision tree divides observations according to predictor conditions and assigns predictions within terminal regions. This can represent nonlinear relationships and interactions without specifying every term in advance. Yet a tree can become unstable: a small data change may alter an early split and much of the subsequent structure. A deep tree with very small leaves can memorize noise. Depth limits, minimum leaf sizes, pruning, and validation help manage complexity. The best training fit is not sufficient evidence that the chosen structure will generalize.
Bagging fits models to resampled training datasets and combines their predictions. Random forests add randomized predictor selection at splits to reduce correlation among trees. Averaging can lower variance when errors are not perfectly correlated, but it does not remove every source of bias or data error. Out-of-bag observations can support an internal assessment in an appropriate independent-data setting. They do not automatically solve time leakage, repeated-customer dependence, or a deployment shift that the resampling scheme has not represented.
Boosting builds an ensemble sequentially, directing new stages toward improving the current model under a specified objective. Learning rate, tree complexity, number of iterations, and other controls affect the bias-variance trade-off and computation. A smaller learning rate is not a universal guarantee against overfitting; it interacts with the number and structure of fitted stages. Use a suitable validation procedure and, where appropriate, early stopping. Keep the final test set for an honest final evaluation rather than repeatedly consulting it until a satisfactory setting is found.
Interpretability tools answer different questions. Global importance summarizes some notion of contribution across a dataset. Local explanations describe a particular prediction under a chosen method. Permutation importance can be affected by correlated features and the distribution of the evaluation data. Partial dependence averages model predictions over varied values of one or more features, which can create unrealistic combinations when predictors are strongly dependent. None of these tools automatically identifies a causal effect or a lawful basis for using a variable.
Operational constraints matter alongside a metric. Compare the complexity, stability, inference cost, maintainability, documentation, and fairness implications of candidate models. A slightly better validation score may not justify a fragile pipeline that cannot reproduce its inputs. Check behavior outside well-supported regions and consider missing or unseen values at deployment. For count, cost, or probability predictions, verify that the objective and output treatment match the target's meaning. Present an ensemble as a fitted predictive tool with limitations, rather than as an inherently superior method or an excuse to stop understanding the insurance problem.