An adoption claim, selection bias, and a twelve-crore scale-up
The state extension department reports that 1,200 of 4,000 rice households in a district have adopted a new short-duration variety. Adopters average 5.9 t/ha and non-adopters 5.0 t/ha. The department claims an 18% yield gain and proposes Rs 12 crore to scale the variety to all 40,000 ha of the district. A review of the baseline shows adopters hold 1.8 ha on average against 1.2 ha for non-adopters; 78% of adopters have tube-well irrigation against 41% of non-adopters; and 65% of adopters had at least three extension visits in the previous year against 22%. The variety matures in 115 days against 135, which allows timely wheat sowing. The district has 55% assured irrigation; the remaining area depends on canal supply that is often delayed. The department wants a report in two weeks.
1. Is the 18% figure a credible causal estimate of the variety's effect? Identify the biases and their likely direction.
Read the worked answer
No. The comparison is between two groups that differ systematically before the variety is considered.
Adopters have 50% more land (1.8 ha vs 1.2 ha), nearly twice the tube-well access (78% vs 41%), and three times the extension contact (65% vs 22%). Each of these independently raises yields and the probability of adoption. Larger farmers can afford seed and take risks; tube wells give timely transplanting and irrigation; extension contact brings knowledge of the variety and of other practices.
The comparison therefore confounds the variety with farm size, irrigation and extension. The likely direction is upward: 18% is probably an overestimate of the variety's true effect, perhaps substantially, because part of the difference belongs to irrigation and management. There is also the short-duration benefit: the 20-day earliness only pays where wheat can be sown on time, which again requires irrigation. Conversely, some adopters may have chosen the variety because their fields were already better, which reinforces the upward bias.
The figure can be reported only as an association among adopters, not as a causal impact.
2. What design would you propose for the two-week deadline, and what would you propose for the longer term? State the data each requires.
Read the worked answer
Within two weeks: a matched comparison restricted to irrigated farms, where confounding is smaller. Match adopters and non-adopters on land size, irrigation source, extension visits and village, using propensity score matching or simple exact matching within strata. Report the matched difference with a confidence interval, and run a falsification check on pre-adoption yields or on a recall question about the previous season. This needs the household listing already held by the department, plus a short phone or field survey of 200-300 households.
Also estimate a difference-in-differences if baseline yield is available from the crop-cutting records: compare the change for adopters and non-adopters, which removes fixed differences between farms. If baseline data are missing, do not invent them.
Longer term: a cluster-randomised encouragement design across 60-80 villages, with extension and seed access randomised and outcomes from crop cutting. Alternatively, use distance to the seed dealer or social-network exposure as an instrument, but test the exclusion restriction, because distance may also proxy for market access.
Report the design and its assumptions to the department, not just a point estimate.
3. What should the department do with the Rs 12 crore scale-up proposal, and why?
Read the worked answer
Do not approve a blanket scale-up to 40,000 ha on the current evidence. Phase it.
First, target the 55% assured-irrigation area, about 22,000 ha, where the short duration can actually be converted into timely wheat sowing. Promoting a 115-day variety on delayed canal command areas risks poor establishment and a yield penalty if canal water arrives late, which would discredit the variety.
Second, allocate part of the budget to seed multiplication and to the extension visits that the data show are correlated with adoption; otherwise the scale-up will reach the same better-off farmers and widen the gap.
Third, reserve 5-10% of the budget for an embedded evaluation: baseline survey, crop cutting in treated and comparison villages, and a follow-up after two seasons. This converts Rs 12 crore from a claim into evidence.
Fourth, report the 5.9 vs 5.0 t/ha figure with the caveat that it is an unmatched comparison, and present the matched estimate as the planning number. If the matched gain is, for example, 8% rather than 18%, the budget and the expected return change substantially. A phased, evaluated scale-up protects both the farmers and the department's credibility.