Choosing an Outcome Measure for a Food-Pantry Nutrition Education Program
A food pantry partners with a dietitian to launch a 6-session nutrition education series for clients. Funding requires reporting 'improved nutrition' at 6 months. The pantry serves about 300 households monthly; about 40 attend at least one session. The funder's template asks for a percentage of participants with improved dietary quality. The dietitian has limited budget, no research staff, and clients who are highly mobile with inconsistent contact information. The board is excited about a 'before-and-after' survey and wants to claim that the program caused improvement.
1. What evaluation design and measures are feasible and defensible here, and what are their limitations?
Read the worked answer
Feasible design: A single-group pre/post design with a validated short dietary screener (e.g., a brief fruit/vegetable or diet-quality screener) at baseline, post-series, and 6 months, plus process measures (attendance, retention). Given mobility and budget, expect substantial attrition; plan for it rather than pretend it away.
Measures: Dietary screener score as the primary outcome; secondary measures such as food-security status, self-efficacy, and pantry use. Keep the instrument short to maximize completion.
Limitations to state plainly: No comparison group, so secular trends, regression to the mean, and self-selection (attendees differ from non-attendees) cannot be ruled out. Attrition may bias results if those who leave differ systematically. A single-group pre/post can support 'participants improved' but not 'the program caused improvement.'
Mitigation: Collect minimal contact data for follow-up, offer multiple completion modes, report response rates and characteristics of completers vs. non-completers, and pre-specify the analysis before data collection.
2. The board wants to claim the program caused improvement. How do you advise them, and what language would you recommend for the funder report?
Read the worked answer
Advise against causal language given the design. Overclaiming is both an integrity problem and a practical one - a funder or reviewer who spots the overreach may discount the whole report.
Distinguish what the data support: 'Among participants who completed both surveys, mean screener scores increased from X to Y' is defensible. 'The program improved nutrition' is not, without a comparison.
Suggested report language: 'Participants who completed baseline and follow-up assessments (n = __ of 40; __% response) showed an increase in [measure] from __ to __. Because the design lacked a comparison group and follow-up was incomplete, results should be interpreted as promising but not causal. Process data show __ sessions delivered to __ participants.'
Offer a path forward: If the funder needs stronger evidence, discuss a stepped-wedge or waitlist comparison across pantry sites, or a matched comparison group, in the next funding cycle.
3. Retention is a known problem. What practical steps would you take to protect data quality without overburdening clients?
Read the worked answer
Reduce burden: Keep surveys short (validated brief screeners), offer paper, phone, and text options, and provide a small incentive consistent with ethics and funder rules.
Improve reachability: Collect two contact methods plus an alternate contact with consent; ask preferred contact mode and timing; use pantry visits as natural follow-up points.
Build tracking into service: Administer follow-up at a regular pantry visit rather than a separate appointment; train volunteers to administer the screener consistently.
Plan the analysis for missing data: Pre-specify how attrition will be handled and reported; avoid complete-case-only reporting that hides who was lost. Report the denominator honestly - 40 attendees out of 300 households is itself a key finding about reach.