Overdispersed Claim Counts: Poisson, Negative Binomial, or Zero-Inflated?
An insurer models annual household claim counts using policy-year records from 12,400 policies. The exposure offset is log(policy-years), and the candidate mean structure includes policy age, region, and a no-claims discount band. A Poisson GLM gives a residual deviance of 18,320 on 12,350 degrees of freedom. The Pearson chi-square statistic is 21,700. Adding a dispersion parameter by quasi-Poisson changes standard errors but not fitted means. A negative binomial GLM has AIC 24,180 versus 24,950 for Poisson, and a zero-inflated Poisson has AIC 24,090, but 14% of policies have zero claims against a predicted 9% under the negative binomial. The actuary must choose a model for pricing and for reserving. One region has only 90 policy-years and a raw claim frequency of 0.31 versus a fitted 0.19.
1. Quantify the overdispersion and compare the three models. Which would you choose for pricing and which for reserving? Justify.
Read the worked answer
The Poisson residual deviance divided by degrees of freedom is 18,320/12,350 = 1.48, and the Pearson ratio is 21,700/12,350 = 1.76. Both exceed 1 by enough to reject the Poisson variance assumption, but the difference between the two ratios suggests the mean structure may also be misspecified, or that many small counts and zeros are distorting the Pearson statistic. Quasi-Poisson only inflates standard errors; it does not change fitted means or produce a proper predictive distribution, so it is not a fix for reserving. The negative binomial lowers AIC by 770 relative to Poisson and directly models overdispersion, which is attractive for both pricing and reserving. The zero-inflated Poisson has a further AIC improvement of 90, but AIC comparisons involving different zero-inflation likelihoods need care, and the predictive check shows the negative binomial underpredicts zeros: 14% observed versus 9% predicted. That is evidence for a zero-inflation or hurdle component. For pricing, I would prefer a negative binomial or quasi-Poisson with robust standard errors for stable relativities, then test a hurdle model for the zero process. For reserving, I would require a model that reproduces the zero mass and the tail, so a hurdle or zero-inflated model with out-of-sample calibration would be preferred over plain Poisson.
2. The small region has 90 policy-years and a raw frequency of 0.31 versus a fitted 0.19. What is the statistical issue and how should the rate be set?
Read the worked answer
The raw frequency has an approximate standard error of sqrt(0.31/90) = 0.059, so a 95% interval is roughly 0.19 to 0.43. The fitted value 0.19 sits at the lower boundary of that interval, but the raw estimate is still very uncertain. Setting the price at 0.31 would treat sampling noise as a regional effect and could overprice the region. The correct approach is shrinkage: fit a hierarchical GLM with a random region effect, or use credibility weighting so the regional rate is pulled towards the pooled mean according to the estimated between-region and within-region variance. The fitted 0.19 should not be ignored, but it should also be checked for model fit and data quality. If the region has genuinely different risk characteristics not captured by policy age, discount band, or other covariates, a random effect will estimate that after shrinkage. The actuary should report the shrunk rate with an interval and avoid using the raw 90-policy-year frequency directly.
3. What role does the log(policy-years) offset play, and what would go wrong if it were omitted?
Read the worked answer
The offset fixes the coefficient on log exposure at 1, so the model is for the claim rate per unit of exposure rather than the raw count. This is essential when policies have different policy-year exposures: a policy with twice the exposure should have twice the expected count if the rate is the same. If the offset is omitted, the model effectively estimates a count mean that absorbs exposure differences, so covariates correlated with exposure will be contaminated. Large-exposure policies will dominate the fitted counts, and the resulting coefficients will not be interpretable as rate relativities. Pricing relativities could be biased, and predicted counts for a new policy would not scale correctly with exposure. The actuary should also check that exposure is measured consistently, that zero-exposure records are excluded, and that the proportional exposure assumption is reasonable; if not, exposure should enter as a covariate or the model should be fitted on rates with appropriate weights.