The Duckweed Claim: Confounded Light and an Overread Null
Two student teams test a garden-center label claim that a new liquid plant food doubles duckweed growth. Team A sets 10 cups on a bench - five with 5 mL of plant food, five with 5 mL of distilled water - but places the five food cups on the side of the bench nearest the window. After 14 days Team A reports 42 ± 3 fronds (food) versus 30 ± 4 (water) and declares the claim supported. Team B uses 20 cups, randomly assigns 10 to food and 10 to water, rotates cup positions daily, and reports 38 ± 11 fronds (food) versus 33 ± 10 (water). Team B's teacher says their result "shows no effect." Both teams used the same duckweed clone, the same 40 W lamp, and the same starting volume.
1. Identify the two most serious flaws in Team A's design and explain how each could by itself produce the 42-versus-30 difference.
Read the worked answer
Flaw 1 - treatment is confounded with light. All five food cups sat on the window side. Duckweed growth is strongly light-limited, so the food group received both the food and (probably) more light. The 12-frond gap cannot be assigned to either variable. Randomizing cup position and rotating daily would break the association.
Flaw 2 - the treatment was not independently replicated. Cups clustered in one location share one microenvironment, so position acts as a single experimental unit; the effective sample size is closer to 1 than to 5. Any bench-scale gradient (light, temperature, drafts) becomes a systematic bias rather than random noise. Team A also never recorded day-0 frond counts, so initial differences cannot be subtracted out, and counting was not blind.
Judgment: the difference is at least as easily explained by light as by plant food. A design that cannot separate its treatment from a known growth driver supports no conclusion about the product.
2. Team B's teacher says their result "shows no effect." Evaluate that claim and state the conclusion the data actually support.
Read the worked answer
The teacher is confusing "no effect detected" with "no effect."
- Standard error of each mean (if ± is SD, n = 10): 11/√10 ≈ 3.5 and 10/√10 ≈ 3.2.
- SE of the difference ≈ √(3.5² + 3.2²) ≈ 4.7, so the 5-frond gap is only about 1.1 SE (t ≈ 1.1, df ≈ 18, p ≈ 0.28).
- That is not significant, but the 95% CI on the difference runs roughly −5 to +15 fronds, i.e. anything from a 15% decrease to a 45% increase.
The data are compatible with the claimed doubling being false and with a real 15% effect. Correct conclusion: no effect was detected at this sample size; the study is underpowered for the effect size of interest. Before saying more, check whether ± is SD or SE (SE would make the two groups clearly different), and compare day-0 counts to confirm the groups started equal.
3. With a budget of 40 cups and 28 days, redesign the experiment so that a 15% effect could be either detected or confidently ruled out.
Read the worked answer
Design changes that buy power without buying cups:
- Two treatments only, n = 20 each. Splitting cups across extra doses is what made Team B weak.
- Measure growth rate, not final count. Count fronds on days 0, 7, 14, 21, 28 and analyze the slope or the paired change per cup. This removes cup-to-cup starting variation, which is a large part of the ±10–11 SD.
- Block by position. Pair cups into 20 adjacent blocks, randomize which treatment goes in each block, and rotate within blocks. Position effects become blocks, not noise.
- Blind the counter (coded cups) to remove scoring bias.
- Pre-specify the effect size and do a power calculation, and report means with 95% CIs rather than a bare p-value.
Arithmetic check: with unpaired final counts (SD ≈ 10), n = 20/group gives SE_diff ≈ 3.2, so a 5-frond effect is only ~1.6 SE - still marginal. The paired/rate analysis is what makes 40 cups sufficient, because it removes the between-cup variance that dominates the raw counts.