The Fertilizer Trial with Two Confounds
An agricultural extension officer in the Terai region tests a new organic fertilizer on wheat. Eight 0.5-hectare plots are selected. Four 'treatment' plots receive the fertilizer at sowing; four 'control' plots receive the standard practice. All plots use the same wheat variety and seed rate. At harvest, treatment yields are 4.2, 4.5, 4.1, and 4.6 tonnes per hectare; control yields are 3.8, 4.0, 3.9, and 4.1. The treatment plots, however, lie on the north side of the field, within 40 metres of the irrigation canal, while the control plots are on the south side, 150–200 metres from the canal. The fertilizer was also applied two weeks earlier than the standard top-dressing date. The officer concludes: 'The fertilizer raises yield by about 0.45 tonnes per hectare, so we should recommend it for the whole district.' A village-level worker questions the conclusion and asks for a redesign before the recommendation goes to the block office.
1. Which of Mill's methods is the officer implicitly using? State the method and identify the condition for causal inference that the design violates.
Read the worked answer
The officer is using the method of difference. Two sets of plots are compared: one has the fertilizer, the other does not, and the yields differ. The method of difference tries to isolate a cause by finding two instances that agree in all relevant circumstances except the one circumstance whose effect is under investigation.
The design violates that requirement. The treatment and control plots do not differ only in fertilizer application:
- The treatment plots are within 40 metres of the irrigation canal; the controls are 150–200 metres away. Water availability is a relevant circumstance and may itself raise yield.
- The fertilizer was applied two weeks earlier than the standard date, so timing also differs.
Therefore the observed difference cannot be attributed to the fertilizer alone. The method of difference has not been properly applied, because the two groups are not matched on the other relevant conditions.
2. Calculate the treatment mean, the control mean, and the observed difference. What can and cannot be concluded from these numbers?
Read the worked answer
Treatment mean: (4.2 + 4.5 + 4.1 + 4.6) / 4 = 17.4 / 4 = 4.35 tonnes per hectare. Control mean: (3.8 + 4.0 + 3.9 + 4.1) / 4 = 15.8 / 4 = 3.95 tonnes per hectare. Observed difference: 4.35 − 3.95 = 0.40 tonnes per hectare, not 0.45 as the officer states. The officer has also made an arithmetic error.
What can be concluded: the treatment plots yielded 0.40 tonnes per hectare more on average in this one field in this one season. That is an observed association.
What cannot be concluded: that the fertilizer caused the difference, or that it will raise yield by 0.40 tonnes per hectare across the whole district. The treatment plots were closer to the canal and received the fertilizer earlier. Either factor could account for part or all of the difference. With only four plots per group, one field, and one season, the result also cannot support a district-wide recommendation. The 0.40 figure is at best an upper bound on any fertilizer effect in this trial, not a clean causal estimate.
3. Propose a stronger trial in four to five sentences. Include randomization, blocking or matching, timing, and at least one measurement that would help separate the fertilizer effect from the irrigation effect.
Read the worked answer
Use at least 20 plots and randomly assign fertilizer or control within blocks defined by distance from the canal, so that near-canal and far-from-canal plots are represented in both groups. Apply the fertilizer to all treated plots on the same date and irrigate all plots according to the same schedule, recording soil moisture before each irrigation. Measure baseline soil organic matter and moisture at sowing, then harvest each plot separately and compare paired treatment-control differences within each block. Repeat the trial for at least two seasons and, if possible, at two or three sites; only then would a district-level recommendation be empirically supported.