Payroll Migration: Fifteen Days, Two Critical Defects, One Pass Rate

Product risk = likelihood × impact.

• SSO login - highest. Likelihood high: new third-party integration, never tested, your configuration and attribute mapping are unproven. Impact high: if login fails, nothing else matters. A vendor certificate covers the vendor's product, not your integration - error handling, MFA, leavers, duplicate accounts and timeout behaviour are all yours to test.
• Tax calculation - highest, different flavour. Likelihood high (complex, changed, historically three times the defect density), impact high (wrong pay, legal and employee-relations consequences).
• Payslip PDF layout - lowest. Cosmetic, contained, cheap to fix post-release.

Evaluating the cuts:

• Dropping tax tests for two 'simple' categories is only safe if you have confirmed those categories run through genuinely separate code. A shared rule engine means the same defect affects them, and 'simple' employees are often the largest population. Reduce if justified, do not eliminate; test the shared calculation once with representative data.
• Skipping SSO is not a risk-based decision - it removes the highest-risk area on the strength of someone else's testing. If time truly runs out, test the login/logout/error paths at minimum and record the residual risk with an owner.

The 83% figure is close to meaningless on its own, and using it as a sign-off argument would be misleading. Three things are hidden inside it:

1. It is 83% of the 96 executed cases - that is 30% of the 320 planned. Coverage is the missing denominator.
2. It says nothing about which tests passed. 83% pass with the high-risk tax cases unexecuted is worse than 60% pass that includes all high-risk coverage passing.
3. Two critical defects are open. A pass rate counts executed cases, not unresolved business-blocking defects.

What to present instead:

• Coverage by risk area (executed/planned for SSO, tax, payslips, interfaces), not overall counts.
• Open defects by severity, with age and owner, especially the two critical tax defects and their fix status.
• Defect trend: are we finding fewer, less severe defects over time, and is the find-fix cycle converging?
• Residual risk list with what will not be tested and the consequence.
• A recommendation with a decision the business owns: go, go with documented risk acceptance and workaround, or delay. Exit criteria agreed earlier should be the frame, and any deviation recorded with the name of who accepted it.

Treat it as an evidence and requirements problem, not an argument. Steps:

1. Make the report objective: exact steps, input data (employee category, gross pay, tax code), expected vs actual amounts, environment and build, and the business rule or regulation that makes the result wrong. Numbers, not adjectives.
2. Check the source of truth. If a requirement, payroll regulation or the vendor's calculation specification states the rule, the defect stands and the design is wrong. If the specification is genuinely silent or ambiguous, you have two items: a defect in the requirements (log it, raise with the BA/product owner) and a clarification needed before the fix.
3. Run it through triage - tester, developer, BA and project manager - where severity (business impact: incorrect net pay, compliance exposure) and priority (urgency of fix) are agreed separately. Severity does not drop because a developer disagrees.
4. Do not close as 'as designed' without a documented decision naming the decision-maker; keep the item open until then, and keep it on the residual risk list.

Given it is critical and in tax calculation, it is a release blocker unless the business formally accepts the risk with a workaround - and that acceptance should be in writing.