Fertilizer, Windowsills, and a Confounded Comparison
Two seventh-grade groups test the claim "Fertilizer makes bean plants grow taller." Group A places 10 bean plants in a south-facing window, waters them 200 mL daily, and adds 5 g of fertilizer each week. Group B places 10 bean plants in a north-facing window, waters them 100 mL every other day, and adds no fertilizer. All plants are the same variety and were started on the same day in the same potting mix. After three weeks, Group A's mean height is 18.4 cm and Group B's mean height is 14.2 cm.
Maya concludes that the fertilizer caused the 4.2 cm difference. Her teacher asks the class to examine the design before accepting that conclusion, pointing out that the two windowsills receive very different amounts of light and that the watering schedules were never matched. Group A's shortest plant is 15.9 cm; Group B's tallest is 16.3 cm.
1. Identify every variable that differs between the groups besides fertilizer, and explain why the design cannot separate the fertilizer effect from these other factors.
Read the worked answer
Differences beyond fertilizer:
- Light - south-facing versus north-facing window, a large and continuous difference in intensity and duration.
- Water volume - 200 mL daily versus 100 mL every other day, roughly four times as much water.
- Water frequency - daily versus every other day, which also changes how consistently moist the soil stays.
Because these changed together with the fertilizer, the groups are confounded: any difference in height could be caused by fertilizer, by light, by water, or by some combination. Nothing in the design lets us separate them.
There is a second, subtler problem: each window is a single location, so window position itself is unreplicated. Even if both groups had been watered identically, a south window's warmth and light would still be tangled with the treatment.
2. The ranges overlap: Group A runs from 15.9 cm up, Group B up to 16.3 cm. What does that overlap imply about Maya's 4.2 cm conclusion, and what additional information should she request?
Read the worked answer
Overlapping ranges mean the two groups are not cleanly separated. Group B's tallest plant outgrew Group A's shortest plant, so "fertilized plants are taller" is not true of every individual - the means differ, but the distributions overlap.
Before drawing any conclusion, Maya should ask for:
- The spread of each group - range, or better, standard deviation - to judge whether a 4.2 cm gap in means is large relative to the variation within groups.
- All individual measurements, so she can see whether one unusually tall or short plant is pulling a mean.
- Whether the same plants were measured by the same person with the same method, since measurement differences of a centimeter or two are easy to introduce.
A mean difference with overlapping ranges is a hint worth following up, not a settled result.
3. Redesign the experiment so a single causal claim about fertilizer would be justified, and state precisely what conclusion the original data do support.
Read the worked answer
Redesign:
- Keep all plants in the same light conditions - one window, or better, rotate positions regularly so no plant gets a permanently favored spot.
- Give identical water (same volume, same schedule) to every plant.
- Vary only fertilizer: 5 g weekly for half the plants, none for the other half.
- Randomly assign which plants get fertilizer, rather than assigning by shelf or group.
- Increase the number of plants per treatment and include a control for handling if fertilizer application involves extra disturbance.
- Measure all plants the same way, on the same days, by the same person or with a fixed ruler setup.
What the original data support: only that the plants in Group A's conditions ended taller on average than those in Group B's conditions. Because light and water varied too, the 4.2 cm gap cannot be attributed to fertilizer.