Two-Supplier Reject Rates and an Underpowered Comparison
A contract manufacturer of infusion sets is qualifying two visual-inspection suppliers for the same connector. Over one quarter, Supplier A shipped 37 rejects in 1,200 inspected lots and Supplier B shipped 26 in 1,200; a two-proportion test returns p = 0.16. Procurement wants to consolidate to Supplier B and has booked €180,000 of annual savings from reducing the supplier base, although B's unit price is 40% higher and B needs six weeks to ramp. Annual volume is 2.4 million units. A reject caught at goods-in costs €3.50 to sort and rework; one escaping to a customer is carried in the risk file at a €22,000 CAPA and field-action estimate. The champion wants a go/no-go decision at Friday's tollgate, the analysis phase has consumed 60% of the project budget, and the team lead has drafted the slide 'no significant difference between suppliers - consolidate to B'.
1. Interpret the p = 0.16 result properly. Construct the 95% confidence interval for the difference in reject rates and state what the data do and do not support.
Read the worked answer
Point estimates: p_A = 37/1200 = 3.08%, p_B = 26/1200 = 2.17%, difference = 0.92 percentage points.
Standard error: SE = √[0.0308(0.9692)/1200 + 0.0217(0.9783)/1200] = √(0.0000249 + 0.0000177) = 0.00652
95% CI: 0.0092 ± 1.96 × 0.00652 = 0.0092 ± 0.0128 → −0.36 pp to +2.20 pp.
What the data do not support: 'no difference'. A non-significant p-value is not evidence of equivalence; the interval still admits a difference of more than two percentage points in A's disfavour and a small difference in B's disfavour.
What the data do support: the observed difference is small relative to its uncertainty, and at 1,200 per arm the test could only reliably detect a difference of roughly 2 pp or more. The experiment was never sized to answer the question being asked.
Report to the tollgate: 'observed 0.92 pp in B's favour, 95% CI −0.36 to 2.20 pp, inconclusive'. Putting 'no significant difference' on the slide and then consolidating to B is a decision made on procurement grounds and mislabelled as a statistical finding - say so explicitly.
2. How large a sample would be needed to detect the observed 0.92 pp difference with 80% power at α = 0.05? What does that imply for a six-week ramp-up decision?
Read the worked answer
Sample size per arm (two-proportion, two-sided):
n = (z_α/2 + z_β)² × [p₁(1−p₁) + p₂(1−p₂)] / d² n = (1.96 + 0.84)² × (0.0308 × 0.9692 + 0.0217 × 0.9783) / 0.0092² n = 7.85 × 0.0511 / 0.0000846 ≈ 4,750 per arm
So roughly 9,500 lots in total, about four times the data collected.
Implications:
- At the current inspection rate the powered study takes about four quarters, not one - it cannot inform Friday's decision or a six-week ramp.
- A larger study is only worth funding if the difference at that size would actually change the decision. Ask that question first: at 2.4 million units/year, a 0.92 pp difference is ~22,000 extra rejects, about €77,000 of sort-and-rework at €3.50. Against a 40% unit-price premium, the defect difference is almost certainly not the deciding term.
- If a defensible answer is still needed cheaply, use a paired or stratified comparison (same lot, both suppliers, same connector family) and a sequential design so the study can stop early if the difference is large.
3. Give the recommendation you would put to the champion, including how to handle the sunk budget and the risk-file cost of escapes.
Read the worked answer
Recommendation: decouple the two decisions.
Consolidation is a commercial decision, not a quality verdict. €180,000 of base savings and a 40% price premium are large enough to decide the sourcing question on total cost; do not dress it up as 'statistically better quality'. Write the slide as: 'quality difference inconclusive at current sample size; consolidation justified on commercial grounds with quality risk managed by the controls below'.
Price the risk explicitly. If B's estimated true reject rate is anywhere in the upper half of the confidence interval, the €22,000-per-escape figure dominates everything. Insist on a goods-in sampling plan for the first three months sized to detect a 2 pp shift (e.g. tightened 100% for the first 10 lots, then 200 lots/week), plus a defined containment and CAPA trigger. That converts an unquantified risk into a costed control.
Sunk cost is irrelevant to the decision. The 60% of budget already spent is the same whether you consolidate or not; only future cost and risk bear on the choice. Say this once, plainly, because it is the argument the team is actually using.
Close the analysis phase honestly. Record the study as underpowered, state the sample size that would answer the question (~4,750 per arm), and hand the powered comparison to the quality engineering function as routine supplier monitoring rather than a Black Belt project.
Do not present p = 0.16 as proof the suppliers are equivalent.