Which Soil Holds the Most Water? A Fair-Test Failure
Ms. Tran's fourth graders ask which soil holds the most water. Six groups each get one soil type - sand, potting soil, or clay - put it in a paper cup with a hole, pour water through, and measure what collects in a clear cup below. Because materials are limited, the sand groups use 30 mL of soil packed down, potting-soil groups use 50 mL loose, and clay groups use 40 mL. Sand groups pour 100 mL and wait 2 minutes; the other groups pour 50 mL and wait 10 minutes. Collected: sand 78 mL, potting soil 22 mL, clay 8 mL. The class concludes that sand holds the most water. One potting-soil group knocked over their cup and re-ran the trial, which gave 30 mL instead of 22 mL.
1. Evaluate the class's conclusion. Which parts are supported by the data and which are confounded? Identify each uncontrolled variable and the direction it pushes the results.
Read the worked answer
The conclusion is wrong twice over.
Error 1 - the operational definition is inverted. 'Holds water' means water retained, not water that passed through. Retained = poured − collected:
- Sand: 100 − 78 = 22 mL retained
- Potting soil: 50 − 22 = 28 mL retained
- Clay: 50 − 8 = 42 mL retained
By the class's own data, clay retains the most, not sand. Sand let the most through, which is the opposite of holding.
Error 2 - the comparison is not controlled. The variables that differ and their bias:
- Amount of soil (30 / 50 / 40 mL): more soil means more pore space, so potting soil and clay are favored over sand on retention.
- Packing: packed sand has fewer pores than loose potting soil, again disfavoring sand.
- Volume poured (100 vs 50 mL): sand received twice the water, so it had far more to retain, favoring sand on retention.
- Wait time (2 vs 10 min): longer drainage means less water collected, which artificially raises the retention score for potting soil and clay.
The variables push in opposite directions, which is why raw numbers cannot settle the question - not even the direction of the final claim is trustworthy.
2. Rewrite the procedure so the comparison is fair, and state exactly how 'holds the most water' should be measured.
Read the worked answer
Operational definition: water retained = volume poured − volume collected in the cup below. Report it as a percentage of the water poured so groups cannot be compared on different starting amounts.
Controlled procedure:
- Same soil volume in every cup - 50 mL, measured in a graduated cylinder, same cup size.
- Same packing method: tap the cup on the table three times, no pressing.
- Same water volume: 100 mL, poured from the same height at the same rate.
- Same wait time: 10 minutes for all cups, with collection cups checked until dripping stops.
- Same water temperature and same soil source.
- Three trials per soil type, averaged, so one odd result does not decide the claim.
Data table: soil type | volume poured | volume collected (trial 1, 2, 3) | mean collected | mean retained | % retained.
Prediction to elicit first: students should commit to a prediction and a reason (particle size, pore space) before running the test, so the result can surprise them and be discussed.
3. What should the teacher do with the knocked-over trial and the 30 mL result, and what does that teach about evidence?
Read the worked answer
Discard the 30 mL as data, and say so out loud. The trial was not run under the protocol - the cup was disturbed - so the number does not measure the potting soil's retention. It is an accident, not a result.
What to do instead: re-run that trial cleanly under the same protocol and record it alongside the other potting-soil replicates. Report in the class notebook that a trial was discarded and why, so the record is honest.
What not to do: do not average 30 in with 22 - that would report a number no trial produced. And do not keep 30 because it 'looks better,' which is the mistake this moment is built to prevent.
The teaching point: evidence has to be produced by the agreed procedure, or it does not count, no matter who produced it or how reasonable it seems. Connect this to the group that re-ran the trial and got a number they liked less - the procedure protects you from choosing the answer you wanted. A short written reflection ('Why can't we use the spilled cup's number?') turns the accident into the lesson, and afterwards the class should re-examine the original sand conclusion with the corrected definition and controlled data, since clay is likely to come out on top.