The Feeding Program That Couldn't Show It Worked
A city runs a 120-day supplementary feeding program in 30 public elementary schools, targeting children identified as wasted. The program provides one fortified meal on school days. The city health office reports that 71% of participating children gained weight, and the mayor's office wants to scale the program to all 90 schools next year using PHP 48,000,000. You are the monitoring and evaluation officer. You have: baseline and endline weight-for-age for 1,850 participants; a list of 900 eligible children in the same schools who did not participate because their parents declined; attendance records for the meals; and school-level data on whether the school had a feeding program the prior year. The budget office asks you to justify or revise the scale-up. You have six weeks.
1. Assess whether the 71% weight-gain figure supports the claim that the program works, and identify the comparison you would construct.
Read the worked answer
The 71% figure describes participants only. It cannot separate the program's effect from growth over 120 days, seasonal food availability, or the attention that comes with being selected. Weight gain alone is a weak outcome because some gain is normal in children.
A defensible comparison is available: the 900 eligible non-participants. Construct a comparison group from them, matched to participants on baseline weight-for-age z-score, age, sex, and school. Then compare the change in z-score, not raw weight, between groups.
Two cautions. First, the non-participants declined, so they may differ systematically from participants in ways that affect nutrition; check baseline characteristics and adjust or report the imbalance honestly. Second, contamination is possible if non-participants share meals or if schools had prior programs. Use the prior-program variable to stratify.
If the matched comparison shows a z-score improvement meaningfully larger than the comparison group's, the program has a defensible effect. If the difference is small, the 71% figure is mostly growth and selection, and the scale-up case must rest on something else.
2. Using the data available, propose the analysis plan and the decision rule for scaling up, including what would make you recommend a partial scale-up.
Read the worked answer
Analysis plan.
- Compute change in weight-for-age z-score per child from baseline to endline.
- Estimate the difference in mean change between participants and matched non-participants, with a confidence interval.
- Add a dose measure: days attended out of 120. If children with higher attendance show larger gains, that strengthens a causal reading.
- Stratify by whether the school had a prior feeding program, to check whether gains are concentrated where the program was new.
- Report cost per child and cost per unit of z-score improvement.
Decision rule.
- Full scale-up if the adjusted difference is positive, statistically distinguishable from zero, and large enough to matter nutritionally, and if cost per unit of improvement is within the budget office's threshold.
- Partial scale-up if effects are positive but concentrated in schools without prior programs, or in children with the lowest baseline z-scores, or if attendance is uneven. In that case, target the highest-need schools and children first and phase the rest.
- Revise before scaling if the adjusted difference is near zero or if attendance is too low to deliver the intended dose.
Budget implication. At PHP 48,000,000 for 90 schools, the per-school cost is PHP 533,000. Compare this to the measured benefit before committing the full amount.
3. What would you do in the six weeks to strengthen the evidence, and what are the limits of what you can claim?
Read the worked answer
Six-week actions.
- Clean and lock the participant and non-participant datasets; document matching rules before looking at outcomes.
- Verify a random sample of 10% of weight records against source forms to catch transcription error.
- Compute attendance distribution; if many children attended fewer than 60 days, report the program as delivered at lower dose than designed.
- Run the matched comparison and the dose analysis, and pre-specify the decision rule in a short memo to the budget office before results are known.
- Interview a small number of parents who declined, to understand whether refusal was random or related to need.
Limits. This is a quasi-experiment, not a randomized trial. Residual confounding from unmeasured household factors remains. Weight-for-age cannot distinguish stunting from wasting, so the program's effect on the specific target group is uncertain. Results apply to this city and this 120-day cycle. Claims should be phrased as "associated with" improvement under a matched comparison, not as proof of causation, and the scale-up should include a built-in evaluation so the next cycle produces stronger evidence.