Forecasting Monthly Claim Counts Through a Reporting-System Change

Ljung–Box and the lag-12 spike. The joint test rejects white-noise residuals, and the rejection is localised: one spike at lag 12 with nothing at lags 11 or 13. That single-lag signature is unmodelled seasonality, not general autocorrelation. With d = 1 and an AR(1), the model has no mechanism to represent a repeating annual pattern, so it treats the seasonal swing as short-run persistence that decays.

Consequence for the point forecast. A model with d = 1 and no seasonal term reverts to a flat level, which is precisely what is reported - 9,150 for every month. For a series with a January–February peak, that flat forecast is wrong in a predictable direction: it will understate January and February and overstate the spring and autumn. The ±14% interval is also unreliable, because it is computed from a model whose residual variance is contaminated by the unmodelled seasonal signal.

The unmodelled level shift. The platform change at month 70 is not in the model. First differencing turns a step into a one-off spike, which the AR(1) then propagates for a few months and then forgets. The practical effect is that the model cannot distinguish “frequency changed because reporting changed” from “frequency changed because risk changed,” so the post-2020 level is carried forward as if it were pure risk signal. That contaminates the 2023 level, not just the seasonal shape.

What this rules out. You cannot present these forecasts. Two independent defects - missing seasonality and an unmodelled structural break - both bias the answer in identifiable directions.

Specification. Two additions, kept separable so each can be tested:

1. Intervention term. A step dummy Sₜ = 0 for months 1–69, 1 from month 70, entered as a regression term with ARIMA errors. The coefficient should recover roughly the 4% reporting effect. If it recovers far more than 4%, the dummy is absorbing a genuine frequency trend and the series needs to be exposure-adjusted (on-levelled) instead.
2. Seasonality. A seasonal term at lag 12 on the non-seasonally-differenced series, e.g. SARIMA(1,1,0)(1,0,0)₁₂ or (1,1,0)(0,0,1)₁₂. A seasonal AR is appropriate when the lag-12 autocorrelation decays gradually; a seasonal MA when it cuts off after lag 12. Given a single sharp spike, the MA form is the first candidate.

Why not D = 1. Seasonal differencing with only eight years of monthly data costs a full year of observations and risks over-differencing, which shows up as a large negative spike at lag 12 in the ACF of the differenced series. Check the ACF/PACF of the seasonally differenced series before committing; if the lag-12 autocorrelation becomes strongly negative, stay with D = 0.

Checks before acceptance:

• Residual ACF/PACF inside the ±2/√n bands at lags 12, 24 and 36, not just at low lags.
• Ljung–Box at 24 lags (12 lags is too short once seasonal structure is present).
• AICc comparison across candidates, holding d and D fixed so the comparisons are on the same differencing.
• Stability of the step coefficient under small changes to the sample window.
• Forecasts should now show a January–February peak and a July bump; a flat forecast again means the seasonal term is not doing its job.

Validation. Hold out the last 24 months and score candidates on RMSE and MAPE (or MASE, which is scale-free and comparable across the raw-count and exposure-adjusted series). Report both the point-forecast error and the empirical coverage of the 95% intervals - a model can have the best RMSE and still produce intervals that cover 70% of outcomes, which would be disqualifying for a committee that plans around the upper bound.

A trap in the backtest. The holdout straddles the platform change, so any model estimated on pre-change data and forecast forward will fail through the break. That failure is informative (it says the model cannot anticipate structural change) but it is not a fair ranking criterion for models that include the step dummy. Structure the backtest so that the intervention is handled consistently: either estimate the dummy on the full sample and validate on the post-change period only, or validate on an artificial break placed earlier in the sample.

Should pre-2020 data be used? Yes - but for the right job. Twenty-eight post-change months is far too short to estimate a stable seasonal shape, so discard 2015–2019 and you lose the very parameter you most need. The defensible decomposition is to estimate the seasonal shape on the full history (seasonality is plausibly stable across the break) and the level on the post-change data, or better, to adjust the historical counts for the reporting change by backfilling late-reported claims from a development triangle. Backfilling is preferable to a dummy because a dummy is a step and cannot represent the ramp during the transition year, and because it produces a series that can be on-levelled to a consistent basis going forward.