Motor renewal pricing: GLM versus gradient boosting
A UK motor insurer has 750,000 policy-years of private car exposure and 12,000 claims. The pricing team fits a Poisson GLM with 18 factors and 6 interactions; the current GLM has a scaled deviance of 0.83 and is used for renewal quotes. A gradient-boosted tree model using the same predictors plus 10 telematics summaries reduces out-of-sample Poisson deviance by 4.2% and improves lift in the riskiest decile from 2.9 to 3.4. However, partial dependence for driver age is non-monotonic between 21 and 25, and the model relies heavily on postcode. The regulator expects pricing to be explainable, fair and proportionate; the insurer has a model risk policy requiring independent validation and ongoing monitoring. The data science team proposes full deployment for renewal pricing next quarter, with a short explainability note.
1. Should the insurer deploy the GBM for renewal pricing next quarter? Recommend a decision and justify it against the model risk policy, predictive gain, explainability and fairness.
Read the worked answer
Recommendation: do not deploy as the sole renewal-pricing model next quarter. The 4.2% out-of-sample deviance reduction is material, but pricing decisions need calibrated risk, not just rank. Validation should show calibration by age, postcode and telematics band, out-of-time performance, and confidence intervals around the deviance gain. The age pattern may be genuine, but a non-monotonic 21-25 effect is exactly where overfitting and proxy effects hide; check partial-dependence stability across folds and whether monotonic constraints change the gain. Heavy postcode reliance raises fairness and proxy-discrimination risk, so test residuals and outcomes for protected and proxy groups and consider removing or constraining postcode. A short explainability note is not enough for the model risk policy: require independent validation, global and local explanations, reason codes, adverse-action logic and monitoring. A controlled challenger pilot alongside the GLM, with capped premium movements and human review of large changes, is proportionate. Full deployment only after validation and fairness sign-off.
2. Design an independent validation and ongoing monitoring plan for the GBM. Specify the tests, metrics and escalation triggers you would require.
Read the worked answer
Validate on an out-of-time holdout by renewal quarter, not a random split; refit the feature pipeline inside each fold. Test discrimination (lift, Gini), calibration (observed/expected by decile, segment and telematics band), stability (PSI/CSI, SHAP stability), fairness (subgroup error, premium impact, proxy tests) and stress scenarios (telematics missing, postcode recoding). Monitor monthly: claim frequency vs predicted, deviance, calibration slope/intercept, PSI, override rate and complaints. Pre-set triggers, for example a calibration ratio outside 0.95-1.05 for two months, PSI above 0.2, or a subgroup premium impact above a governance threshold, requiring review or rollback. Independent validation should reproduce the data extract, model artefact and score distribution. Document model limitations and the role of actuarial judgement. Report to the model risk committee, not only pricing.
3. If the GBM is used only to rank policies for underwriting review, not to set price, how does your recommendation change?
Read the worked answer
Ranking for review is lower-risk but not risk-free. If the score only prioritises underwriting review, the insurer still must show that review is fair, explained and human-led; the score must not become a de facto decline. Use it to allocate specialist review, with a sample of low scores checked to measure false negatives, and capture reviewer overrides. The GBM may be acceptable sooner because pricing harm is reduced, but fairness and proxy tests are still needed and customers should not face adverse action without a clear reason. Marketing use would be a different purpose and should not be assumed.