The Plant That Keeps Saying 'Nothing Has Changed'
A mid-sized food manufacturer has run the same risk register for three years. The register lists 42 risks scored on a 5x5 matrix using likelihood and impact. The risk committee meets quarterly and spends almost the whole meeting reviewing the top ten risks by score. Last year a recall caused by a supplier's undeclared allergen cost GBP 2.1m, yet the register had rated 'supplier quality failure' as low likelihood (2) and medium impact (3), because the supplier had never failed before. The head of quality wants to add the supplier to a watch list. The finance director argues that the matrix is objective and that re-scoring after the event would be hindsight bias. Two other incidents in the past 18 months were also rated below the top ten before they occurred. The CEO has asked whether the framework itself is the problem.
1. Diagnose why the scoring framework failed to surface these risks. Is the finance director's hindsight-bias objection valid?
Read the worked answer
The framework failed for structural reasons, not because of bad luck.
1. Likelihood was estimated from observed frequency only. With no prior failures, the supplier scored a 2. For low-frequency, high-consequence events, historical frequency is a poor estimator: the absence of an event is not evidence of low likelihood. A structured technique (fault tree, bow-tie, or expert elicitation with calibrated ranges) would have exposed the wide uncertainty.
2. Impact was under-scoped. The GBP 2.1m recall cost was scored as a '3' because impact was read as direct cost to the plant, not including regulatory action, lost contracts, and brand damage. Scoping impact consistently across financial, legal, safety, and reputational dimensions is part of the framework design.
3. The matrix was treated as objective. A 5x5 matrix is a communication device, not a measurement instrument. Scores are judgements; presenting them as objective hides the assumptions behind them.
4. Attention was allocated by rank, not by tolerance. Reviewing only the top ten by score means anything scored below the cut-off is effectively unmanaged, regardless of whether it breaches a tolerance.
5. No feedback loop. Three below-top-ten incidents in 18 months is a pattern. A functioning framework would trigger recalibration of the scoring method, not just re-scoring of individual risks.
On hindsight bias: the objection is partly valid but misapplied. It would be wrong to say 'we should have known this supplier would fail.' It is not hindsight bias to conclude that the method systematically under-weights low-frequency, high-consequence exposures. That is a design critique supported by three independent outcomes, and it can be tested prospectively against the next 20 supplier risks.
2. Recommend three specific changes to the framework that address the diagnosis without abandoning the existing register.
Read the worked answer
Change 1 - Separate routine and non-routine risk assessment. Keep the 5x5 matrix for high-frequency operational risks where frequency data is meaningful. For low-frequency, high-consequence exposures (supplier failure, single-source dependency, key-person loss), use a scenario-based assessment: define the event, estimate a plausible worst case, and assess whether the organisation could absorb it. This prevents the 'no prior event = low likelihood' trap.
Change 2 - Score impact on a defined, multi-dimensional scale. Publish an impact table with explicit thresholds for financial loss, regulatory breach, safety, and reputational harm, and require the assessor to record which dimension drove the score. Apply a consistent time horizon (e.g., 24 months). This makes scores comparable and auditable, and would have caught the recall's full cost.
Change 3 - Replace top-ten review with tolerance-based review. Define risk appetite and tolerance per category. The committee reviews every risk that breaches tolerance, plus a rotating sample of the rest, plus all risks flagged as 'low likelihood, severe consequence.' Add a standing agenda item on near-misses and incidents that were scored below tolerance before occurring, with a requirement to explain the discrepancy.
Supporting change: record the assumptions and evidence behind each score so that re-scoring after an event is a documented learning exercise rather than an accusation of hindsight. This directly answers the finance director's concern.