A discount bug that three passing tests missed
An online shop applies 10% off orders, for members only, when the order total is £50.00 or more. The website advertises '10% off orders of £50 or more'. A member reports being charged full price on an order of exactly £50.00. The developer's three tests all pass: £100 member (expect £10.00 off), £10 member (expect nothing), £100 non-member (expect nothing). The code stores totals as floating-point numbers and the condition reads if total > 50. A second complaint says a member received a discount on an order shown as £49.99. The tester has one afternoon, no formal requirements document beyond the website wording, and the shop takes about 900 orders a day.
1. Explain why the three existing tests passed while the business rule is broken, and list the test cases you would add, with expected results.
Read the worked answer
Why they passed: all three sit in the middle of an equivalence class - a large member order, a small member order, a large non-member order. None is anywhere near £50, so none can distinguish > from >=. They do confirm the 10% multiplication and the membership check work, which is why they look reassuring. Passing tests prove only that the cases tested pass; the untested decision is the boundary.
Cases to add (pence, member unless stated):
- 4999 → expect 0
- 5000 → expect 500 - this is the failing test the customer reported, and it should be written first and watched failing
- 5001 → expect 500 or 500.1 depending on the stated rounding rule (see below)
- 0 → expect 0 (empty basket)
- 5000, non-member → expect 0, confirming membership is still required at the threshold
- 4999.995 → the float case: the basket may be stored as 4999.9999999 or 5000.0000001 depending on how it was summed, which is how an order displayed as £49.99 can pass a threshold test and produce the second complaint.
That is about six cases: enough for the decision, small enough for one afternoon.
2. Decide whether the reported behaviour is a code defect, a requirements ambiguity, or something else, and set out the order of work for the afternoon with reasons.
Read the worked answer
Classify first, because the two complaints have different causes.
- Complaint 1 (£50.00 gets nothing): the published wording 'of £50 or more' is inclusive, so
total > 50contradicts the rule the customer was shown → a code defect. The condition was evidently written from the developer's mental model, not from the text customers see. - Complaint 2 (£49.99 gets a discount): not the same bug and probably not a bug at all in the everyday sense - floating-point storage means a total near £50 can compare as above it. The display rounds to £49.99 while the stored value is a hair over 5000. Two issues; fix them separately or a regression test for one will mask the other.
Order of work:
- Write the failing test (member, 5000, expect 500) and watch it fail - so the fix is known to be the cause of the pass.
- Confirm the rounding rule for 5001 with whoever owns the shop wording; encoding a guess as an expected value creates false authority.
- Change the comparison to inclusive and move money handling to integer pence.
- Re-run the boundary set plus the original three; keep all of them.
- Record the resolution in the bug report so the next person does not re-litigate it.
What not to do in one afternoon: do not replay the 900-order daily catalogue or rewrite the pricing module. The decision has a small set of inputs; test the inputs.
3. The developer says the tests were written to match the code, which is normal practice. Evaluate that position.
Read the worked answer
It is normal engineering practice and it is exactly how this bug survived. Tests written from the code encode the code's assumptions, so a test suite written that way can be fully green while contradicting the customer-facing rule. The suite becomes a description of what the program does, not of what it should do - useful for catching regressions, useless for catching a wrong decision.
Here the specification was available and short: one sentence on the website, '10% off orders of £50 or more'. That is enough to derive 5000 → 500 without any formal requirements document, and it is the only source the customer will quote when complaining.
Practical compromise for a one-afternoon task:
- Derive threshold and membership tests from the published wording and label them as business rules.
- Derive additional internal tests (rounding, negative totals, float edge) from the code and label them as characterisation tests.
- Where the two disagree, as here, the published wording wins and the code changes - or the wording is changed deliberately by the business and the test follows.
Where no specification exists at all, write down the decision you are assuming; an unwritten assumption is what a bug report becomes later.