Defect Debt in a Payments Release
A payments platform team ships every two weeks. Over the past quarter, escaped defects rose from 3 to 14 per release, and mean time to resolve a production incident went from 40 minutes to 4 hours. The team attributes the change to a new feature area with complex currency rounding rules. Their test suite has 620 automated tests, but the build takes 55 minutes and developers have started running only a subset locally before merging. A test-coverage report shows 81 percent line coverage, which leadership cites as proof of quality. Two of the last four incidents traced to rounding edge cases that no test covered. The team has no shared definition of what "done" means for a rounding change, and code review is optional for hotfixes.
1. Leadership points to 81 percent coverage as evidence quality is fine. How do you reframe the problem?
Read the worked answer
Line coverage measures which code executed during tests, not whether the tests assert the right behavior. The two incidents came from rounding edge cases with no test at all, which is entirely consistent with 81 percent coverage. Coverage is a weak signal and a dangerous one when it is used to close a conversation.
I would reframe around escaped defects and detection time, which are outcome measures leadership already tracks. Fourteen escaped defects per release and a sixfold increase in time to resolve are the real story. I would add a targeted measure: for each escaped defect, was there a test that would have caught it, and if not, why not.
The build time is part of the causal chain. A 55-minute build pushes developers to run subsets locally, so integration problems surface late. I would treat build duration as a quality metric with a target, and prioritize test-suite parallelization or splitting so the full suite runs fast enough that running it is the path of least resistance.
Finally, I would propose a rounding-specific test charter: property-based or table-driven tests for currency edge cases, written with the domain expert, so the untested area gets deliberate attention rather than more line coverage.
2. Hotfixes bypass code review and there is no shared definition of done for rounding changes. What do you do?
Read the worked answer
Both are process gaps that convert small problems into recurring ones. A hotfix path with no review is how a rounding error becomes a second incident.
I would not simply mandate review, because a mandate under production pressure will be bypassed. Instead I would make the safe path fast: a lightweight paired review for hotfixes, one reviewer with a short checklist focused on the changed calculation and its test, and a follow-up ticket to add the missing regression test within the next sprint. Track hotfixes that ship without a test and make that visible.
For the rounding definition of done, I would convene the team with the domain expert and write it explicitly: boundary values tested, currency and locale cases enumerated, an independent calculation cross-check, and a review by someone other than the author. Post it where stories are refined and use it as an acceptance gate.
I would also run a short retrospective on the two incidents to identify which step in the definition of done would have caught them, so the new standard is grounded in evidence rather than imposed.