The Audit Finding That Reads Like a Waterfall
An internal audit of a general insurer's claims platform found three things: the calculation change released in March had no documented traceability from requirement to test; release notes were missing for two of the last nine production releases; and one release had no recorded approval. Audit has rated the finding "major" and recommends that the teams must produce a full requirements specification and obtain formal change approval before every release. The platform runs three teams on two-week sprints with automated regression coverage of 82% of critical claim paths, a definition of done that includes peer review and an automated test run, and a change advisory board (CAB) that meets weekly with an average ten-day lead time from submission to approval. The CIO has asked the delivery lead to comply with the audit recommendation by next quarter to avoid escalation to the board risk committee. The teams argue that a full specification before every release would double lead time and that they already test more than they did under waterfall.
1. How do you respond to the audit recommendation without reverting to a specification-first process? Set out the reasoning and the specific artefacts you would propose.
Read the worked answer
Separate the control objective from the prescribed remedy. Agree the first, negotiate the second.
The objective audit actually cares about is: changes are tested to a defined standard, approved by someone with authority, and the evidence survives scrutiny months later. The prescribed remedy - a full specification before every release - is one way to achieve that, not the only way, and it is the most expensive one available here.
Concede the real gaps. They are genuine and cheap to close:
- No traceability from requirement to test - a real gap.
- Missing release notes on 2 of 9 releases - a real gap.
- No recorded approval on one release - a real gap, and the most serious.
These are evidence-capture failures, not process-maturity failures. A team that tests thoroughly but records nothing fails an audit exactly like a team that does not test.
Propose lightweight, automated evidence:
- Traceability from acceptance criteria to automated tests - each story's acceptance criteria carry test identifiers, and the pipeline emits the mapping automatically. This produces the traceability audit wants without a separate specification document.
- A generated release record attached by the pipeline: artefacts included, test run output, approver identity, timestamp. Evidence that is produced by the tooling cannot be skipped under delivery pressure.
- A release decision record - a 15-minute template capturing what changed, residual risk, and who accepted it.
Argue against the remedy on audit's own terms: a full specification before every release adds lead time without adding control, because the control is testing and approval. Offer a joint test: pick two releases and let audit sample the evidence. Give them a success criterion they can verify.
2. The CAB has a ten-day lead time and the teams run ten-day sprints. What changes do you propose to change management, and how would you get the CAB to agree?
Read the worked answer
Diagnosis: a ten-day approval lead time inside a ten-day sprint means every approval decision is made on stale information, and teams either wait or route around the board. Both outcomes are worse than the risk the CAB exists to manage.
The board is a batch-and-queue control applied to small, frequent, lower-risk changes. It was designed for large, infrequent releases. The mismatch, not the board's intent, is the problem.
Proposal - risk-tiered change classification:
- Standard / pre-approved: low risk, reversible, covered by automated regression and behind a feature flag. Approved by the product owner plus tech lead, no CAB submission, sampled monthly by change management.
- Normal: customer-visible or data-affecting changes. CAB or a named delegate, with a 48-hour decision commitment so the queue cannot silently lengthen.
- Emergency: unchanged, existing process.
Getting agreement: the CAB's actual fear is being blamed for a failure it did not see. So do not ask for trust - ask for a 90-day pilot on two teams with pre-agreed measures: change failure rate, lead time to approval, and number of unrecorded approvals. Publish them monthly. Offer an explicit rollback: if change failure rate rises above the current baseline, the delegation is withdrawn without argument.
Reinforce the argument with batch size: releasing smaller increments weekly rather than one large fortnightly batch reduces the risk carried per change. Smaller changes, more often, with automated evidence, is a stronger control position than fewer, larger, less-visible releases.
Also report to the board risk committee on control effectiveness - evidence coverage, failure rate, time to remediate - not on document counts.