Mold or Shift? A Confounded Defect Investigation
A plastics plant moulds a housing on two tools. Defect rate for the past month is 4.2% against a 2.0% target. Last month's data: Mold A produced 7,900 parts with a 2.6% defect rate; Mold B produced 4,100 parts with a 6.9% defect rate. A chi-square test on mold gives p = 0.002. On the strength of that, the production manager has requested $85,000 to replace Mold B. What the data extract also shows: Mold A ran almost entirely on shifts 1 and 2, while Mold B ran almost entirely on shift 3, because of a scheduling rule that puts the newer tool on nights. The resin lot changed mid-month, humidity in the plant ranged from 38% to 62% RH, and 100% visual inspection is split between two inspectors with no attribute agreement study on file. Nobody recorded which inspector saw which part. The plant manager wants a decision within three weeks.
1. The production manager says p = 0.002 proves Mold B is bad. Evaluate that conclusion and state what the test does and does not establish.
Read the worked answer
What the test establishes There is an association between tool and defect rate in this month's data. The probability of seeing a difference this large by chance alone, if the two tools truly had the same rate, is about 2 in 1,000. That is real. It is also all the test says.
What it does not establish
- Causation. Chi-square is a test of independence, not a causal model.
- That tool is the driver. Mold and shift are almost perfectly confounded - Mold B is essentially the night shift. Any shift effect (staffing, supervision, ambient temperature, humidity, inspection rigour) is baked into the "Mold B" number. The design is a quasi-experiment with a structural confound that no amount of statistical significance can remove.
- That replacement will fix it. If the real driver is shift-3 process settings, a new tool will run on nights and inherit the same problem.
- That the measurement is sound. Two inspectors, no agreement study, and no linkage of inspector to part. If inspector 2 is stricter and inspected more of Mold B's output, some of the gap is measurement, not process.
Also worth checking Expected counts and whether parts are independent. Parts from the same shot are correlated; a chi-square that treats 12,000 parts as independent overstates significance.
A defensible statement: "Tool is associated with defect rate, but the effect is inseparable from shift and inspection in the available data." An $85,000 capital request is not justified on that.
2. What analysis and what new data do you need to disentangle tool from shift within the three-week window?
Read the worked answer
Analysis on existing data
- Stratify by shift. Compute defect rate by tool within shift. Even with the imbalance, shift 1 and 2 give you Mold A only, so the comparison is thin - but it tells you whether Mold B's rate is uniform across the nights it ran or spikes with particular conditions.
- Stratify by date and resin lot. Plot defect rate by day for each tool. If Mold B's rate jumped when the resin lot changed, you have a second confound competing with the tool explanation.
- Correlate with humidity using daily records. If defects track RH above roughly 55%, the story changes entirely.
- Mantel–Haenszel or logistic regression with tool, shift, resin lot and humidity as terms, acknowledging that the tool–shift collinearity will produce wide confidence intervals. Report the uncertainty; do not hide it.
New data - the decisive step Run a short crossed trial: each tool on each shift, at least two shifts each, with the same resin lot, in randomized order, blocked by day. Even 2 shifts × 2 tools × 3 days gives 12 cells and breaks the confound.
Measurement first Run a 30-part attribute agreement study with both inspectors on the same parts and report kappa. If kappa is below 0.7, recalibrate the visual standard and the defect catalogue before the crossed trial, or the trial will reproduce the same ambiguity.
Communication Tell the plant manager explicitly: the three-week decision is achievable, but it is a decision about what to investigate, not a decision to buy a tool.
3. The crossed trial runs and Mold B is still worse on every shift, though the gap narrows from 4.3 to 2.0 points. What do you recommend, and how does that change the $85,000 request?
Read the worked answer
What the result means The gap survives crossing, so tool is a real contributor - but roughly half the original gap was shift, not tool. Two facts follow: replacing the tool would deliver less than the manager expected, and shift 3 has a problem of its own that will not be fixed by a new tool.
Recommendation
- Do not present replacement as the only option. With the tool effect now ~2.0 points, ask the toolroom and the moulding engineer to examine B's cooling circuit, venting and shut-off surfaces for a maintenance-level cause. A refurbishment costing a fraction of $85,000 may recover most of the gap. Quantify: 2.0 points on 4,100 parts a month is about 82 defects a month; multiply by scrap and rework cost per part to get the ceiling on any spend.
- Fix the shift-3 problem separately. The residual shift effect was worth ~2.3 points in the original data. That is now a clear, independently addressable target - supervision, settings standardization, start-up checks - and it is cheaper than capital.
- Re-sequence the capital request to cover only the irreducible tool contribution, with the payback calculation attached, and present it alongside the shift action.
- Verify with a confirmation run on the fixed tool across both shifts before closing the improvement.
Method point for the team The original analysis was not wrong, it was incomplete. The lesson is the value of crossing a suspected factor with the confounded nuisance variable before spending money - a two-week trial against an $85,000 decision.