Backtesting a Rates Desk VaR: Eleven Exceptions in 250 Days
A rates trading desk at a global bank runs a 1-day 99% historical-simulation VaR. Over the last 250 trading days the desk recorded 11 losses exceeding reported VaR; 8 of them fell inside a single three-week window around a surprise central-bank decision. Average reported VaR over the window was USD 42m and the worst single-day loss was USD 96m. A parallel hypothetical-P&L backtest, which strips out fees, intraday trades and reserve movements, produced 5 exceptions over the same period. The desk head argues the model is sound because 'the losses were one event, not eleven', and proposes a 1.6x volatility overlay for the current regime instead of a recalibration. The supervisor applies the Basel traffic-light framework.
1. Quantify the backtesting result. Is 11 exceptions in 250 days consistent with a correctly calibrated 99% model? Show the test and state the traffic-light zone and the capital consequence.
Read the worked answer
Test. If the model is correct, the exception count X over n = 250 days is Binomial(250, 0.01):
- E[X] = np = 2.5
- sd = sqrt(np(1-p)) = sqrt(2.475) = 1.573
- z = (11 - 2.5) / 1.573 = 5.40
A Poisson(2.5) approximation gives P(X >= 11) at roughly 0.00007, i.e. about one year in fourteen thousand. The result is not marginal.
Zone and consequence. Basel's traffic light treats 10 or more exceptions in 250 days as the red zone: plus-factor 1.00 on the base multiplier of 3.00, so the capital multiplier becomes 4.00 - a one-third increase in the VaR-based charge - alongside a supervisory presumption that the model is deficient. The magnitude evidence points the same way: a USD 96m loss against a USD 42m VaR is 2.3x the stated 99% bound.
2. The desk head argues that because the exceptions clustered into one event, the model is fine. Assess that argument together with the 11-versus-5 split between actual and hypothetical P&L.
Read the worked answer
Clustering does not rescue the model; it indicts it.
- The binomial test assumes independent exceptions. Clustering violates that, so the nominal p-value is optimistic - but the direction of the inference is the opposite of what the desk head claims. An 8-in-3-weeks run is exactly the signature of a model whose volatility input is stale or regime-blind. A correctly specified model produces exceptions with no visible serial dependence; a runs test on the exception sequence would reject independence here.
- Clustering therefore points to a calibration failure: the model did not respond when the risk regime changed.
The 11 vs 5 split isolates a second, different problem. The 6-exception gap is not market risk at all - it is fees, intraday positioning, new deals and valuation/reserve moves that an end-of-day VaR cannot see. That is a coverage and P&L-attribution failure, not a calibration one.
So the desk has two distinct deficiencies: volatility that failed to update, and a VaR scope that excludes intraday and valuation P&L. Neither is bad luck, and fixing one does not fix the other.
3. Evaluate the proposed 1.6x volatility overlay as a remediation. What would you require before accepting it?
Read the worked answer
It is a level fix, not a model fix.
- It is backward-looking and procyclical: 1.0x before the shock, 1.6x only after the loss has already been taken - capital bites exactly when it is least useful.
- A single scalar applied to the whole book assumes every risk factor scaled by the same factor. If the miss came from one curve, one option position or one currency, the overlay either over-penalises the rest of the book or still fails to cover the factor that moved.
- It does nothing about the coverage gap revealed by the 11-versus-5 comparison, which is arguably the more serious finding.
What I would require before accepting it as an interim control:
- A rule-based trigger and a defined exit path - not a discretionary overlay that can be dialled up or down at will.
- A parallel run demonstrating it would have produced 4 or fewer exceptions over the 250 days without overstating VaR on the quiet days.
- A dated remediation plan toward a properly recalibrated model, in practice a 97.5% expected shortfall with stressed calibration.
- Explicit acknowledgement that the overlay does not substitute for the red-zone capital add-on, and that it is supervision of an interim state rather than acceptance of the model.