Stratifying a Persistent Defect Rate
A Green Belt analysing invoice errors at a finance shared-service centre has 6 months of data: 4,800 invoices, 312 errors (6.5%). Stratification shows: Team A 4.1% (n=2,400), Team B 9.8% (n=2,100), Team C 3.0% (n=300). Errors by invoice type: standard 5.1%, complex 12.4%. Team B handles 70% of complex invoices; Team A handles 25%; Team C handles 5%. A Pareto of error reasons shows 'missing PO reference' 41%, 'wrong cost centre' 28%, 'duplicate payment' 12%, other 19%. The manager wants to retrain Team B. A colleague suggests a chi-square test on team vs error.
1. Is 'Team B is the problem' a supported conclusion? Show the reasoning.
Read the worked answer
Not yet - Team B's rate is confounded with invoice complexity.
Expected split if complexity alone drove errors: if complex invoices error at 12.4% and standard at 5.1%, and Team B has 70% complex, its expected rate is roughly 0.7×12.4% + 0.3×5.1% ≈ 10.2% - very close to the observed 9.8%. Team A, with 25% complex, expects ≈ 0.25×12.4% + 0.75×5.1% ≈ 6.9%, but observes 4.1% - i.e. Team A is better than complexity predicts.
So the apparent team effect largely disappears once you stratify by invoice type. The correct next step is a stratified analysis (compare Team A vs B within complex invoices only, and within standard only), or a logistic regression with team and complexity as factors.
Chi-square caution: a single 3×2 chi-square on team vs error would be significant and misleading - it tests association, not cause, and ignores the confounder. If used, it must be on stratified tables, and with Team C n=300 the expected cell counts may be low, so exact methods or pooling may be needed.
Recommendation: do not retrain Team B on this evidence. Retraining a team whose rate is explained by case mix wastes effort and damages trust.
2. Given the Pareto, what does the data actually point to, and what would you investigate next?
Read the worked answer
The Pareto is dominated by missing PO reference (41%) and wrong cost centre (28%) - together 69%. These are process/input failures, not obviously skill failures:
- Missing PO reference usually means the PO was not raised or not linked before invoice arrival - an upstream process or system-validation issue.
- Wrong cost centre usually means a default, a stale master data value, or a dropdown that permits invalid entries.
Next investigations:
- Cross-tabulate error reason against invoice type and team - is 'missing PO' concentrated in complex invoices (where PO may not be required)? If so, the 'error' may partly be a definitional issue.
- Check whether the system permits an invoice to be posted without a PO reference. If it does, that is a poka-yoke opportunity (Improve), not a training issue.
- Sample 20 'wrong cost centre' invoices and trace how the value was entered - default, copy-forward, or manual.
Framing for the manager: the data says the problem is concentrated in two specific failure modes that look systemic; retraining addresses neither. Propose a targeted improvement on PO validation and cost-centre defaults, measured against the same 6.5% baseline.