The Replicated Fertilizer Trial on Sloping Plots
A high-school research class tests whether a slow-release nitrogen fertilizer raises maize yield. Eight plots lie along one slope: the four upper plots drain toward the four lower plots. Students assign the upper four plots to fertilizer and the lower four to no fertilizer, then measure stalk height at week 6 and grain mass at harvest. Stalk heights (cm) are 141, 138, 145, 139 for fertilized plots and 132, 136, 129, 134 for controls. Grain masses overlap almost completely, and the control plots received runoff from above. The class concludes the fertilizer works because the height difference is consistent. Their teacher asks them to defend the design, the measurement choice, and the conclusion before the results are entered in a regional science fair.
1. Evaluate the experimental design. Identify the specific flaw, explain how it could produce the observed result, and propose a revised layout using the same eight plots.
Read the worked answer
The design confounds treatment with position. All fertilized plots sit upslope, so they differ from controls in soil depth, drainage, and nutrient runoff received, not only in fertilizer. Because water and dissolved nutrients move downhill, the lower control plots may be systematically poorer, which alone could create the ~8 cm height gap. The flaw is lack of randomization and of blocking by position.
A defensible redesign pairs plots by elevation: within each of the four slope bands, randomly assign one plot to fertilizer and one to control. This makes each elevation band a block, so slope effects are balanced across treatments. Alternatively, alternate treatments along the slope in a predetermined random order and record elevation as a covariate. Either way, the treatment must not be perfectly correlated with position. The current layout cannot separate a fertilizer effect from a hillslope gradient, so the conclusion is unsupported regardless of how consistent the heights look.
2. The height difference is consistent while grain mass overlaps. Which endpoint should drive the conclusion, and what does the mismatch imply about the measurement plan?
Read the worked answer
Grain mass is the endpoint tied to the claim about yield; stalk height is a proxy that can respond to nitrogen without translating into harvestable grain. A consistent height gap with overlapping grain mass means the treatment may affect vegetative growth only, or that the grain measurement is too noisy at n=4 to detect a real difference.
With four plots per group, variability is poorly estimated and a single outlier dominates. The mismatch should prompt three actions: (1) treat grain mass as the primary outcome and height as secondary; (2) report variability (range or standard deviation) rather than only group means, since overlap of ranges signals weak separation; (3) increase replication or take multiple subsamples per plot to reduce measurement noise. Concluding that fertilizer 'works' from height alone substitutes an easier measurement for the one the question actually asks about, which is a validity error, not merely a precision problem.
3. A classmate says the result is proven because the fertilized heights never dropped below any control height. Assess this reasoning.
Read the worked answer
Complete separation of two small groups is suggestive but not proof. With n=4 per group, the probability of perfect separation by chance under random assignment is small (about 1 in 70 for a one-sided split of eight ranks), which is meaningful but still a single unreplicated trial on one slope in one season. Perfect separation also cannot distinguish treatment from the position confound described above, because position was never randomized.
'Proven' overstates what one observational-style comparison can support. The appropriate language is that the pattern is consistent with a fertilizer effect and warrants a randomized, replicated follow-up. Proof in science means convergence of independent, controlled evidence, not a single clean-looking data set. The class should report the result as preliminary, state the confound explicitly, and avoid causal language until a blocked or randomized design is run, ideally across two seasons or sites.