Twelve Exceptions: Backtesting a KES 2.4bn VaR Model
A fund's risk unit reports a one-day 99% parametric VaR of KES 50m on a KES 2.4bn portfolio, built from 60% equities (22% annual volatility), 40% bonds (9% annual volatility) and an assumed correlation of 0.15. Over the last 250 trading days there were 12 losses exceeding the VaR estimate. In the two stress episodes, equities and bonds fell together and realised correlation reached 0.65. Bond marks come from a dealer matrix updated weekly, and 18% of the bond book has not traded in 60 days. The board asks whether the KES 50m figure can be relied upon for the limit framework and the ICAAP submission.
1. Reproduce the VaR calculation and confirm whether the arithmetic is right.
Read the worked answer
Portfolio volatility:
sigma_p^2 = w_e^2 sigma_e^2 + w_b^2 sigma_b^2 + 2 w_e w_b rho sigma_e sigma_b = (0.6^2)(0.22^2) + (0.4^2)(0.09^2) + 2(0.6)(0.4)(0.15)(0.22)(0.09) = 0.017424 + 0.001296 + 0.001426 = 0.020146
sigma_p = 14.19% annual
Daily scaling: 14.19% / sqrt(250) = 0.897%
VaR: 2.326 x 0.00897 x 2,400m = KES 50.1m
The arithmetic is correct. That is precisely the problem: the number is right and the model is wrong. A board that hears "the maths checks out" will stop asking the right question.
2. Backtest the model formally and identify what is driving the failure.
Read the worked answer
Test: expected exceptions at 99% over 250 days = 2.5. Observed = 12.
Standard deviation of the exception count = sqrt(250 x 0.01 x 0.99) = sqrt(2.475) = 1.573
z = (12 - 2.5) / 1.573 = 6.04
This rejects the model at any conventional confidence level. Four to five exceptions would already put the model in the regulatory amber zone; twelve puts it in the red.
Decomposition:
- Correlation. Rerun with rho = 0.65: sigma_p^2 = 0.017424 + 0.001296 + 2(0.24)(0.65)(0.0198) = 0.017424 + 0.001296 + 0.006178 = 0.024898; sigma_p = 15.78%; VaR = 2.326 x (0.1578/15.81) x 2,400 = KES 55.7m. That is only about 11% higher - it explains a couple of exceptions, not twelve. The 0.15 assumption is wrong, but it is not the main culprit.
- Normality. Real return distributions have fat tails. At 99%, parametric VaR understates systematically; the observed clustering is consistent with tail risk, not with an unlucky sample.
- Stale marks. With 18% of the bond book unmarked for 60 days, and weekly matrix marks elsewhere, measured volatility is suppressed and losses appear as jumps on revaluation - the exceptions are real but arrive late.
Conclusion: the failure is structural - a normal-distribution, single-correlation model applied to a book with illiquid and fat-tailed exposures.
3. Should the board rely on the KES 50m figure? Recommend what replaces it and what to report to the regulator.
Read the worked answer
No. A model with 12 exceptions in 250 days must not be the basis for limits or capital without adjustment.
Replace with:
- Expected shortfall at 97.5% using historical simulation on at least three years of data, including the two stress episodes. ES is sub-additive and does not depend on a normal tail.
- Correlation stress: report VaR alongside a stressed version at rho = 0.65, which gives about KES 56m on the same method - recognising this is still not enough on its own.
- Illiquidity add-on: for the 18% untraded, use a proxy series (an index or a traded comparable) and add an explicit exit-cost add-on of, say, the observed bid-ask plus a 20-day liquidation horizon scaled by sqrt(20). Weekly matrix marks must move to independent weekly verification with aged-mark escalation.
- Scenario and stress testing as the primary limit tool: equities -20%, bonds +150bp yield, correlation 0.65 run together, reported to the board monthly.
Reporting: disclose the 12 exceptions, the z-statistic, the diagnosed causes and the remediation timetable. Concealing or re-basing the backtest - for example re-running with a 250-day window that happens to exclude the stress episodes - is exactly the behaviour supervisors treat as a governance failure, and it would invalidate the ICAAP narrative even if the capital number looked better.