Acceptance Criteria That Cannot Fail
Northwind Insurance is six sprints into rebuilding its claims intake portal. The product owner writes user stories with acceptance criteria such as 'the claim form must be intuitive' and 'the system must respond quickly'. Of 42 stories accepted so far, 11 have generated post-release defects, including one where a duplicate claim was created when the browser back button was used. The business analyst maintains a 60-page requirements document from the old system, but developers rarely open it. A new regulatory rule - claims over €10,000 need a second approver - was found during sprint review and is now being retrofitted. The team uses two-week sprints, has one tester, and the product owner wants to move to 'lighter documentation'. Testers are asked to sign off stories in the last two days of the sprint.
1. Explain why the current acceptance criteria are failing as a test basis and describe a concrete technique the team should adopt in the next refinement session.
Read the worked answer
The criteria fail because they are not testable. 'Intuitive' and 'quickly' cannot produce a pass/fail result, and neither is tied to a concrete example, so the tester invents a personal interpretation in the last two days of the sprint. The back-button defect is a symptom: the rule about duplicate claims was never elicited, so no test could target it.
Adopt example mapping in refinement. For each story, capture the rules, then one concrete example per rule, then turn the examples into acceptance criteria in Given/When/Then form. Unresolved questions become explicit items for the product owner or compliance, not assumptions.
For the duplicate-claim behaviour, a decision table is more useful than prose. Conditions such as submit versus back-and-resubmit, existing claim identifier, and time window expose combinations the team has not discussed. For non-functional criteria, replace 'quickly' with a measurable statement, for example a 95th-percentile response under two seconds at 500 concurrent users on the agreed reference environment.
This also gives the single tester a defensible basis for sign-off and a source of regression checks.
2. The €10,000 second-approver rule is discovered late. What should the test analyst do to limit the impact, both for the current sprint and for the requirements process?
Read the worked answer
Contain first. Write a backlog item with explicit acceptance criteria covering the threshold boundaries (€9,999.99, €10,000.00, €10,000.01), the roles allowed to approve, the prohibition on self-approval, the audit trail, and handling of claims already in progress. Those boundaries are where defects concentrate; testing only 'over €10,000' would miss off-by-one errors.
Second, perform impact analysis. Trace the rule to existing stories, screens, data model and reports. Identify which already-accepted functionality must change and schedule targeted regression around claim submission, approval workflow and notifications. Do not rely on the existing suite, because it was built without this rule.
Third, fix the process rather than only the code. Ask in the retrospective how a regulatory rule escaped until review: was compliance absent from refinement, or was the checklist incomplete? Add a regulatory question to the definition of ready and invite compliance to example mapping. Update the traceability record so the rule can be shown to be tested, which also supports audit.
3. The product owner wants to drop the old 60-page requirements document entirely. Assess this proposal.
Read the worked answer
Do not discard it blindly. The document is not a good specification, but it is a source of implicit business rules, non-functional constraints and history that no one has yet expressed as examples. The late €10,000 rule is evidence that not all knowledge has been mined.
Treat it as a reference to be harvested, not as the primary test basis. Work with the business analyst to extract decision tables, state transitions, validation rules and regulatory constraints, then validate them with the business and convert them into examples and automated checks. Record which sections have been covered and which are obsolete.
Lighter documentation is acceptable if the knowledge is captured somewhere durable: acceptance criteria, example sets and automated tests. Deleting the document without extraction transfers unknown risk to production. If it is retired, give it an owner, a version and an archive location so that future regulatory questions can still be answered.