Thirty-One High Risks and Twelve Days
Northwind Mutual is replacing its claims-handling system. The test manager asked four business stakeholders to score 58 product risks on likelihood (1–5) and impact (1–5). The result: 31 risks scored 20 or more, every one labelled 'High', and the instruction to the team is 'test all High risks to the same depth'. You have 12 working days of system test. Two of the High risks are: R14 - a claim is approved twice when the assessor double-clicks Submit (impact 4, likelihood 4); R27 - the statutory fraud-report file is rejected by the regulator's portal (impact 5, likelihood 2, scored low because 'the old system never had a problem' - although the old system produced the file by hand and the new one generates it from a new data model).
1. The risk register is not supporting prioritisation. Explain what has gone wrong in the assessment and what you would change before the schedule is fixed.
Read the worked answer
What has gone wrong
- The scale has collapsed. 31 of 58 risks at ≥20/25 means 'High' covers more than half the product, so it discriminates nothing. A risk level that everything shares cannot drive test effort.
- Impact is being used as a scoring habit, not a consequence. If nearly every impact is 4 or 5, the anchors were never defined. Impact should be calibrated against concrete consequences: money lost per event, legal or regulatory penalty, harm, reputational damage, number of affected users.
- Likelihood is being inferred from the wrong system. R27 was scored 2 on the operational history of a legacy system that produced the file manually. Likelihood of a defect should reflect defect-proneness of the new solution: novelty of the data model, number and maturity of interfaces, complexity, developer experience, frequency of change, and defect history in comparable modules - not the uptime of the thing being replaced.
- Product and project risks are mixed. If schedule or skills risks are in the same 58, they compete with product risks for the same score.
What I would change
- Re-anchor the impact scale with written examples per level before anyone re-scores.
- Re-derive likelihood for the new build using defect drivers, with the technical team, not only business stakeholders.
- Force differentiation - for example, cap 'High' at roughly the top 15% and require a documented rationale for each High.
- Treat the assessment as iterative: re-score after each test cycle as evidence arrives.
2. With 12 days available, how would you differentiate test depth for R14 and R27? Be specific about technique, coverage and what you would cut.
Read the worked answer
Risk level should drive both the extent of testing and the choice of technique, not just the running order.
R14 (double approval) - a transaction-integrity fault in a stateful workflow. One happy-path script will not find it. I would:
- Model the claim lifecycle as a state transition diagram and derive tests for the illegal transition 'Approved → Approved'.
- Add tests around idempotency: double-click, browser back-then-resubmit, two assessors on the same claim, retry after a timeout.
- Use exploratory/error-guessing sessions at the UI boundary, because the trigger is timing-dependent.
- Check the database constraint or service-level guard that should make the second approval impossible - a test that only exercises the UI cannot tell you whether the business layer is protected.
That is deliberately more than one test case per risk, because the failure mode is a sequence, not a value.
R27 (regulator file rejection) - impact is at maximum and I consider likelihood understated, so this also gets deep coverage: a decision table over the file's mandatory-field and format rules, negative tests with malformed and boundary data, and at least one submission to the regulator's real test portal. A file that is structurally valid but semantically wrong is the expensive failure.
What I would cut. Risks in the mid band get nominal coverage: one representative case per equivalence partition, boundaries only, no combinatorial expansion. Areas where the same technique has already found nothing across two cycles get a single regression pass. The 12 days should buy depth on a handful of risks, not equal slices of 31.
3. Test execution starts in three days and the register will not be fully re-scored in time. What risk-based activities must continue during execution, and who decides?
Read the worked answer
Risk-based testing does not stop at planning; the register is a live control instrument.
During execution I would:
- Track residual risk, not just test progress. A risk is only reduced when tests covering it have run and passed. '80% executed' tells you nothing if the 20% untouched is the top of the register.
- Use defect clustering as evidence. A cluster of defects in the file-generation module raises R27's likelihood retrospectively - I would add tests there and pull effort from a quiet area.
- Re-score on evidence. Where a risk's tests passed convincingly, its likelihood falls and its remaining budget can be released. Where a risk's tests failed repeatedly, it rises and gets more depth.
- Escalate rather than decide alone. Changing a likelihood is a technical judgement an analyst can propose; changing an impact that drives regulatory or financial exposure is a business decision. The product owner or business sponsor owns impact; the test manager owns the resulting reallocation of effort.
- Record accepted risk explicitly. Any risk left under-tested at the end must appear in the test summary report as a residual risk with a named acceptor - not disappear silently because the days ran out.