Mulch, Mud, and a Confounded Hillside
A middle school science club wants to know whether mulch reduces soil erosion on a campus slope. They mark two 4 m × 10 m plots on the same 12° hillside, separated by 3 m. Plot A is left bare; Plot B is rototilled to 10 cm and covered with 5 cm of wood chips. Plot A sits directly uphill of Plot B, and the downhill edge of A drains toward B. Runoff traps collect sediment after each storm. Over six weeks three storms deliver 22, 9, and 41 mm of rain. Traps recover 1.8 kg of sediment from A and 0.4 kg from B. The club concludes that mulch cuts erosion by about 78% and writes the claim on a poster for the science fair.
1. Identify two confounding variables in this comparison and explain how each could produce or mask the observed difference. Then state one design improvement that addresses the most serious problem.
Read the worked answer
Two confounds, and how each could create the result
- Tillage. Plot B was rototilled, which breaks aggregates and loosens surface soil. On its own, tilling should increase erosion in B, so this confound works against the mulch effect rather than explaining it. It still means the plots did not start from identical conditions, so the comparison is not clean.
- Plot position and drainage. A sits uphill and drains into B. Sediment eroded from A can be trapped in B's collector or deposited as a fan, inflating B's 0.4 kg and making mulch look better than it is. B may also receive run-on water, which changes infiltration and surface sealing.
A third problem is that each treatment exists only once, so plot identity is completely confounded with treatment.
Improvement: replicate plots in pairs, till all plots identically, and divert upslope runoff around every plot with a berm or border so no plot receives water from another.
2. Write a testable hypothesis for the mulch effect and a prediction that would distinguish the mulch explanation from the plot-position explanation. Describe the design changes needed to test it.
Read the worked answer
Testable hypothesis: On this slope, with identical soil preparation and no run-on between plots, a 5 cm wood-chip mulch reduces sediment yield per millimeter of rainfall compared with bare soil.
Competing prediction that separates the explanations: If mulch is the cause, then a bare plot placed in the low position (where B now sits) should still lose substantially more sediment than a mulched plot in that same low position. If position or run-on is the cause, the low bare plot will behave like B and lose little.
Design changes:
- Use at least three replicate pairs; randomly assign which member of each pair is mulched.
- Rototill all plots, then mulch only half.
- Install berms and a diversion trench so runoff cannot cross plot boundaries.
- Report sediment mass per unit area per storm, not just a seasonal total, because one 41 mm storm dominates the mass.
- Record runoff volume so sediment concentration can be separated from water yield.
3. The club reports a 78% reduction. Evaluate that number: what does it correctly describe, and what causal claim can it not support?
Read the worked answer
The arithmetic: (1.8 − 0.4) / 1.8 = 0.778, so 78% is the correct ratio for these two collectors.
Why it is not an effect size for mulch: The number describes the difference between two particular plots, not a causal effect of mulch. Three things prevent the causal reading:
- Confounding - tillage, position, and upslope run-on differ between the plots.
- No replication - n = 1 per treatment, so there is no way to estimate plot-to-plot variability or uncertainty.
- Unnormalized mass - the 41 mm storm probably supplied most of the 1.8 kg, and per-storm yields, runoff volume, and sediment concentration are not reported.
What to say instead: "In this unreplicated trial, the mulched plot yielded 78% less trapped sediment, but the design cannot separate mulch from tillage and plot position." A defensible claim needs replicated, hydrologically isolated plots and a mean with a confidence interval.