The Review That Found Nothing
A solution has been in production for eight months. Its design document was approved at inception and has not been revised. The process runs nightly and has had three incidents in the last quarter: one caused by an application upgrade that changed a button's position, one caused by a new exception type the design never anticipated, and one caused by an operator manually editing a queue item. Each incident was fixed by a developer directly in production, with the change recorded only in the developer's own notes. The operations team cannot say which version of the object is currently deployed. A scheduled design review is due, and the sponsor asks what it should cover, given that 'nothing has gone wrong that we didn't fix'.
1. Challenge the sponsor's framing that 'nothing has gone wrong'. What does the incident history actually indicate about the solution's governance?
Read the worked answer
The framing is wrong on its own terms. Three incidents in a quarter is not 'nothing has gone wrong' - it is a pattern. And 'we fixed them' describes recovery, not control. The relevant question is not whether incidents were fixed but whether the solution is being maintained in a controlled way.
What the pattern shows.
- The application-upgrade incident shows there is no change-detection process linking the target application's release cycle to the solution's test cycle. The object was brittle to a UI change that was knowable in advance.
- The unanticipated exception shows the design's exception model is incomplete and is being extended reactively. That is normal in itself, but it should feed back into the design, and here it did not.
- The manual queue edit shows there is no control preventing operators from altering work items outside the process - a data integrity gap, not an operator error.
The governance failure underneath. All three were fixed directly in production with no change record and no version traceability. That means: the approved design no longer describes the running solution; there is no way to roll back to a known-good version; and the next developer has no reliable starting point. The absence of recorded problems is being mistaken for the absence of problems.
2. Design the agenda for the review and specify the artefacts and decisions it should produce.
Read the worked answer
Purpose of the review. Not to re-approve the original design, but to re-establish the link between the documented design and what is actually running, and to put maintenance under control.
Agenda.
- Current-state reconciliation. Establish which object and process versions are deployed, by inspecting the production environment directly rather than relying on notes. Output: a version register.
- Incident review. Walk the three incidents against the design. For each, ask: was this foreseeable, was the response controlled, and what design change should follow? Output: a design change list.
- Exception model review. Compare the exceptions the process now handles against those in the design document. Output: an updated exception catalogue.
- Change control. Agree how production fixes are authorised, recorded, tested, and versioned going forward, including a rollback position. Output: a change-control procedure and an owner.
- Operational controls. Address manual queue editing - who may do it, under what authorisation, and what audit trail results. Output: an access and audit rule.
- Application change watch. Agree how upgrades to the target application are notified to the solution owner. Output: a named contact and a pre-upgrade test step.
Artefacts the review must produce. A revised design document reflecting reality, a version register, an updated exception catalogue, a change-control procedure with a named owner, and a dated action plan with owners for each gap.
Decision to put to the sponsor. The solution needs an ongoing owner accountable for design integrity, not just developers who fix things. Without that, the next review will find the same drift, larger.