NCT Tester Competence Assessment study materials
NCT Tester Competence Assessment study resources

Four Defects, Two Days, One Release Veto

NCT Tester Competence Assessment · Put your knowledge to work in context.

All case studies

Four Defects, Two Days, One Release Veto

Corvus Energy sells invoicing software in six EU markets. Two days before release, triage has four defects. D1: for Estonian VAT-compliant invoices (about 1,100 invoices per month, 2% of revenue), a rounding rule produces a one-cent discrepancy where the net amount ends in .005; the Estonian tax authority requires invoice totals to reconcile to the cent. D2: the dashboard header misspells the company name, visible to every user on login. D3: 'export to PDF' crashes for invoices with more than 400 line items; three customers, all large logistics firms, hit this about twice a week and work around it by splitting the invoice. D4: at month-end peak the invoice list page takes 14 seconds to load against a 3-second target; no crash, but all 6,800 users are affected for roughly two days each month. The developer argues D2 has the highest priority 'because everyone sees it', and the product owner wants to ship with all four deferred because 'none of them stop invoicing'. You hold the test manager's veto on the release recommendation.

1. Assign severity and priority to each defect and justify the assignments, keeping the two concepts clearly separate.

Read the worked answer

Severity is impact on the system and its data. Priority is the order in which the business wants it fixed. They usually correlate and sometimes invert - which is exactly what is happening here.

  • D1, rounding: severity high - wrong financial data is persisted, the invoice does not reconcile to the cent, and a tax authority is involved. Priority critical: it is cheap to fix now and expensive later, because correcting already-issued invoices is a data remediation exercise with a legal dimension. The 2% revenue share and 1,100 invoices per month do not lower severity; they bound the blast radius.
  • D2, misspelling: severity low - cosmetic, no functional or data impact. Priority low to medium: highly visible, so it damages confidence, but it is a patch-release item. This is the case the developer has confused: visibility is not impact.
  • D3, PDF crash: severity medium to high - a functional failure with no data corruption, and a workaround exists. Priority medium: three customers, but large, strategic ones; customer-facing commitment may raise it above D4.
  • D4, month-end performance: severity medium - degraded but usable, no data loss. Priority high if month-end falls within days of release, low if it is weeks away. This is the clearest example of priority depending on timing rather than on the defect alone.

The developer's argument ranks by exposure; the product owner's ranks by 'does it stop invoicing'. Neither is a priority model - one ignores impact, the other ignores legal and data consequences.

2. Recommend a release decision with conditions. Set out what evidence is missing and who must own each accepted risk.

Read the worked answer

Recommendation: conditional no-go, driven entirely by D1.

D1 - block the release until fixed. A rounding fix is typically a small, testable change. Conditions: fix applied; rounding verified across all six markets, not just Estonia; a query run to establish how many invoices have already been issued with the discrepancy, because that determines whether data remediation is needed. Owner of the residual risk if it ships unfixed: compliance or legal, not the product owner alone.

D2 - ship, patch later. Document it in the release note as a known cosmetic issue with a fix date. Risk owner: product owner.

D3 - ship with a communicated workaround. Conditions: the workaround is confirmed by the three affected customers, support has a script, and the fix is scheduled. Risk owner: product owner with account management.

D4 - ship only if month-end is more than a week away, with performance monitoring in place and a rollback plan; otherwise it becomes a release blocker by timing. Risk owner: operations.

Evidence I do not have yet and would demand before signing:

  • Whether D1 affects invoices already issued, and how many.
  • The actual distribution of line-item counts, to size D3 rather than rely on three reports.
  • Performance measurements at realistic month-end concurrency, not a developer laptop.
  • Whether any of these defects share a root cause in the rounding or formatting layer.

On the veto. I would exercise it only for D1 unfixed and without a compensating control. Using it on D2 or D4 would convert a technical judgement into an obstruction, and the release decision record - defects, conditions, owners, dates - is the artefact that makes the recommendation stick.

3. A fifth defect was reported once by a customer, could not be reproduced, and was closed. It has now been reopened with a second report. Describe how you would manage it and what it implies for your testing.

Read the worked answer

Manage it as an information problem, not a 'works for me' problem. Reopen it and move it to 'awaiting information' rather than leaving it as closed or marking it fixed. Then gather, from both reporters: environment and browser or client version, locale and timezone, the data involved (invoice identifiers, amounts, currency), exact steps and timestamps, whether anything unusual preceded it (month-end batch, concurrent edits, an import), and any logs.

Attempt reproduction deliberately. Replay with data matching the reported characteristics rather than with generic test data. Vary the conditions most likely to make a defect intermittent: concurrency, data volume, ordering, locale, and timing between the UI and the batch process. Two independent reports raise the probability that it is real and condition-dependent, which is exactly the class of defect that disappears under casual reproduction attempts.

Track occurrence, not just status. Log each report with a timestamp so the rate becomes visible. A defect that recurs twice in a quarter is a different risk from one that recurs weekly.

What it implies for testing. First, the defect report template is missing environment and data capture - a second report of the same issue is the price of that gap. Second, closing on 'cannot reproduce' treats absence of evidence as evidence of absence; the status should have been 'cannot reproduce yet, insufficient information'. Third, I would run time-boxed exploratory testing around that area with the reported data profile, since scripted regression against standard data is precisely what would have missed a condition-dependent failure. If a root cause is found, it becomes a permanent regression test; until then it stays open with a named owner.