The Significant 1.9 Minutes
Two lines build the same product family. Over eight weeks, Line A averaged 42.7 minutes cycle time (n = 120, s = 5.2) and Line B averaged 40.8 minutes (n = 45, s = 5.4). A two-sample t-test returns p = 0.038, and the team lead proposes copying Line B's layout onto Line A. Two facts surface at the review: Line B runs 85% standard configurations while Line A runs 70% custom, and Line B had a new quick-change fixture installed two weeks into the study period. Shift mix also differs - Line A carries most of the Monday and post-holiday volume. The steering committee meets Friday and wants a recommendation.
1. Interpret p = 0.038 in this specific situation. What does it license, and what does it not license?
Read the worked answer
What it means If the two lines truly had identical mean cycle time, data this far apart (or further) would occur about 3.8% of the time by chance. That is a statement about the data conditional on a hypothesis - not the probability that Line B is better, and not a measure of how much better it is.
What it licenses: that a difference of roughly this size is unlikely to be pure sampling noise, assuming the test's conditions hold.
What it does not license: copying the layout. Three reasons.
- Practical significance is unaddressed. The gap is 42.7 − 40.8 = 1.9 min, about 4.4% of cycle time. Set against a customer tolerance or a takt requirement, that may be noise-level. A p-value answers "is there a difference?", never "is the difference worth acting on?"
- Precision is thin. Pooled s ≈ 5.25, SE = 5.25 × √(1/120 + 1/45) = 5.25 × 0.175 ≈ 0.92. The 95% CI is 1.9 ± 1.8, i.e. roughly 0.1 to 3.7 minutes. The interval nearly touches zero and is wide enough to contain both "irrelevant" and "meaningful." Report the interval, not the p-value, to the committee.
- Assumptions are shaky. Cycle time in a job shop is typically right-skewed; a t-test on skewed data with n = 45 can misstate p. Unequal group sizes (120 vs 45) mean the test is dominated by Line A, whose variance is a mix of custom and standard work - heteroscedasticity is likely. Confirm with a normality check and a Mann-Whitney or bootstrap as a cross-check before leaning on 0.038 at all.
2. Design the analysis that would separate the layout effect from the confounders. Be explicit about what you would stratify or block on and what you expect to see.
Read the worked answer
Confounders to remove, and why each matters
| Confounder | Mechanism | Handling |
|---|---|---|
| Product mix (A: 70% custom vs B: 85% standard) | Custom work is inherently slower - this alone could create a 1.9-min gap with no layout effect | Stratify: compare A vs B within standard and within custom; report the layout contrast only where both lines overlap. Expect the gap to shrink or vanish in the standard stratum. |
| Quick-change fixture on B at week 2 | B's post-week-2 data are contaminated - the fixture, not the layout, may be the cause | Split B's data into pre- and post-fixture; plot cycle time over time by line. If B's mean steps down at week 2, the fixture is the candidate cause, and it is transferable to A without changing the layout at all |
| Shift and day-of-week mix | Monday/post-holiday volume spikes on A; queueing effects inflate cycle time | Block by shift and day-type; run the comparison within matched blocks, or include them as factors in a two-way model. Check whether the gap survives on non-Monday days |
| Time trend / seasonality | Eight weeks of drift can masquerade as a line difference | Plot both lines on the same time axis before any grouped comparison |
Order of work
- Rebuild the comparison as stratified (by product configuration) and blocked (by shift/day-type), with time-series plots first.
- Test the layout contrast within strata; pool only if the effects are homogeneous.
- Separately test the fixture change and quantify its effect size with a CI.
- Check for a Simpson's-paradox reversal: if overall B looks faster but A is faster within every stratum, the aggregate is a mix artifact.
What would change my mind: a gap that holds inside the standard-configuration stratum, survives shift blocking, is present in both the pre- and post-fixture periods, and has a lower CI bound above the practical-significance threshold.
3. Give the recommendation for Friday. State the decision, the evidence basis, and what would make you revisit it.
Read the worked answer
Recommendation: do not authorize copying the layout. Authorize two targeted, reversible actions.
What to tell the committee
- The observed gap is 1.9 min (95% CI ≈ 0.1–3.7 min), which is statistically detectable but not yet shown to be either layout-caused or practically decisive. Comparing 120 to 45 observations makes small differences easy to "find."
- Two of the three largest differences between the lines - product mix and the new fixture - are not the layout. Rolling out the layout would bundle all three and make the result uninterpretable and un-replicable.
Actions
- Evaluate the quick-change fixture on its own. It is cheaper, faster, and reversible. Pilot on one Line A station with matched pre/post data.
- Re-run the comparison stratified by configuration and blocked by shift, with a pre-specified practical-significance threshold (e.g., 5 min or 10% - whichever the capacity plan requires). Set that threshold now, before seeing results.
- Publish the CI, not just the p-value, in the committee minutes.
Conditions that would reverse the recommendation
- The gap holds within the standard-configuration stratum, survives shift blocking, appears in both the pre- and post-fixture periods, and its lower CI bound exceeds the practical threshold.
- Line A's custom-work disadvantage is shown to be a scheduling/batching artifact rather than inherent.
Judgement point: a p-value below 0.05 is a licence to keep investigating, not a licence to rebuild a line. The cost asymmetry - a week of analysis versus a layout change on a live line - makes the order of work the whole decision.