UAT at 82 Percent
Atlas Retail's order management release is five days away. The test summary shows 1,148 of 1,400 planned test cases executed (82%), 94% of executed cases passed, and 11 open severity-1 defects. Two severity-1 defects affect payment reconciliation: under a specific refund sequence, the ledger can be off by the refund amount until the nightly batch. The exit criteria approved at project start require 100% of severity-1 defects closed and a 95% pass rate on executed cases. The business owner wants conditional sign-off because the next retail peak starts in ten days. The QA lead has not tested the new tax-calculation rules in the last two regression cycles. The operations team can run a manual reconciliation for up to 200 refunds per day. Last peak volume was 1,300 refunds per day.
1. Should the release receive sign-off? Justify the decision using the test evidence and operational constraints.
Read the worked answer
The release should not receive unconditional sign-off. The approved exit criteria are not met: 11 severity-1 defects remain open, and the pass rate is 94% against a 95% threshold. More importantly, the two payment reconciliation defects create a financial control weakness, and the proposed manual workaround has insufficient capacity. Operations can reconcile 200 refunds per day, while the last peak volume was 1,300 per day. Under peak conditions the workaround would cover less than one sixth of the exposure, so it is not a credible compensating control.
The 82% execution rate also matters. The untested 18% is not a random sample: the QA lead has not tested the new tax-calculation rules in the last two regression cycles, which is a high-risk area for an order management release. A 94% pass rate on executed cases can look acceptable while hiding defects in the least-tested areas.
The defensible decision is to withhold sign-off, or grant a conditional release only if scope is reduced so that the defective refund sequence and tax rules are not used in production. The business owner's peak date is a real constraint, but it does not convert an unmet exit criterion into evidence.
2. What verification evidence and governance would be required before any conditional release?
Read the worked answer
A conditional release needs evidence that the condition is real and enforceable, not a verbal promise. The QA lead should complete risk-based testing of the tax-calculation rules and the refund sequence, then publish results against the acceptance criteria. The defect triage should classify each severity-1 defect by customer impact, workaround, and detection point, and should identify which defects are release-blocking versus scope-limiting.
The exit criteria themselves should be changed through governance, not ignored. The change should state the revised threshold, the rationale, the residual risk owner, and the duration of the condition. The operations team should provide a capacity plan showing how refunds will be controlled if volume approaches 200 per day, including a stop-sell or manual-review trigger. A rollback plan and hypercare period should be agreed before go-live.
Finally, the business owner and the accountable risk owner should sign the residual risk explicitly. The test summary, defect list, traceability to requirements, and workaround capacity should be attached to the sign-off. That makes the decision auditable and prevents the conditional release from becoming an informal waiver that is never revisited.