The Kalvardi Language Program Review
Kalvardi is a village of 2,300 in a highland district. In 1990, 41 per cent of schoolchildren spoke the Kalvardi language at home; by 2022 the figure was 14 per cent. Three years ago the district introduced two-way immersion: 90 minutes a day, six teachers trained, and elders paid a small stipend as volunteers. Since then, home use of Kalvardi among K–3 pupils has risen from 14 to 22 per cent on an annual parent survey (n ≈ 180), national-language test scores are unchanged (a difference of 1.2 points, within the reported margin of error), absenteeism has fallen 4.3 percentage points, and 11 families have moved back to the village. Some parents complain the program "holds kids back." A consultant's report says the flat test scores "prove the program does not work." Renewal costs $140,000 a year, about 0.9 per cent of the district budget, and the board votes next month.
1. Evaluate the consultant's claim that the flat test scores "prove the program does not work."
Read the worked answer
The claim fails on measurement and on logic.
Measurement: the program's stated goals are language maintenance, attendance and community continuity; national-language test scores measure none of them. A flat score is consistent with success on the program's own terms. Worse, two-way immersion typically produces a temporary dip in national-language scores before transfer effects appear - three years is early.
Logic: "unchanged" is not "harmed." A difference of 1.2 points within the margin of error is a null result, and a null cannot prove absence of effect unless the study had power to detect one. With a small district sample, a real effect of several points could easily be missed. Absence of evidence is not evidence of absence.
Meanwhile the program's other outcomes - attendance up 4.3 points, 11 families returning - are real and are not addressed by the report at all. The consultant has answered an easier question than the one the board asked. The defensible reading is: no detected cost in national-language attainment, plausible benefits elsewhere.
2. Propose an alternative explanation for the rise in home use from 14 to 22 per cent, and describe a design that could test it.
Read the worked answer
Alternative explanation - composition, not behaviour. Eleven families moved back to the village because the program exists. Those families are plausibly Kalvardi speakers already, so the share of pupils speaking Kalvardi at home rises without any existing family changing what it does. The 8-point jump could be largely a denominator effect plus self-report bias: parents who chose the program may over-report home use, and the survey wording may have shifted.
Testing it:
- Track the 11 returning families separately and recompute the rate excluding them. If the rise collapses, composition explains it.
- Add a comparison village without the program to absorb regional trends.
- Replace or triangulate self-report with recorded home conversations rated by independent speakers, and with teacher and elder assessments.
- Keep the instrument identical across years, and note that with n ≈ 180 a single-year estimate carries a 95 per cent margin of roughly ±5.6 points, so an 8-point move is near the noise threshold.
No single method settles it; convergence across methods is the standard.
3. Some parents say the program "holds kids back." Weigh this against the other evidence and state what you would recommend.
Read the worked answer
Weigh magnitude, distribution and reversibility. The complaint is real testimony and should not be dismissed, but it is a claim about cause: that time in Kalvardi displaces national-language learning. The test data do not support that mechanism - scores are flat, not falling, and remain within error. The complainants may also be reasoning from an assumption that a minority language crowds out the majority one, whereas bilingual transfer normally works the other way; that is an ethnocentric framing of a pedagogical question.
The attendance gain matters because absenteeism is a proximate cause of underachievement - a 4.3-point fall is plausibly worth more than the disputed test difference. But the program also has distributional risk: if a small group of pupils is struggling, an average can hide it.
Recommendation: renew for three years at $140,000 (about $580 per pupil per year, cheap against the alternatives), with three conditions - anonymised reporting of the 11 returning families, individual progress data for pupils flagged by parents, and a comparison village. Renewal with evaluation, not renewal on faith.