Attendance, Outcomes, and a Funder Deadline
Bright Steps After-School Program operates at three elementary schools with a $600,000 annual budget funded by a foundation grant and parent fees. It serves 240 students in grades 3–5; 58% attend at least 120 days, below the grant's 75% target. The program's logic model lists improved reading comprehension as a long-term outcome. Last spring, teachers reported that 64% of participating students improved one reading level, but there was no comparison group, no pre-test, and attendance data were missing for 30 students. Staff turnover is 45% annually. The foundation requires a year-end impact report in six months and has hinted that renewal depends on 'strong outcomes.' The program director wants to report the 64% as proof of success and add a new robotics component to attract students. The board asks you to review the evaluation plan and recommend next steps.
1. What conclusions can the program reasonably draw from the 64% teacher-reported improvement figure?
Read the worked answer
The program can report the 64% as a descriptive finding: among students for whom teachers reported a reading level, 64% improved one level. It cannot claim that the program caused the improvement or that the program is proven effective. There was no pre-test, no comparison group, and no standardized measure; teacher reports may be inconsistent across schools. Attendance was low, with only 58% meeting the 120-day threshold, so many students did not receive the intended dose. Missing data for 30 students weakens the denominator and may bias the result if those students differed systematically. The program also cannot call the figure proof of success to the foundation without overstating the evidence. A defensible report would describe the finding, state limitations, explain attendance and staffing challenges, and present it as preliminary evidence to be tested with a stronger design. The board should resist the director's framing because overclaiming risks credibility and renewal more than an honest improvement story.
2. Recommend a six-month evaluation and program improvement plan that is feasible and credible.
Read the worked answer
In the next six months, the program should establish a baseline and improve data quality rather than launch a new component. First, define the outcome clearly: reading comprehension measured by a consistent, validated or locally standardized assessment at pre and post. Second, track attendance daily by student and site, with a target and early-warning follow-up for students missing sessions. Third, if feasible, create a comparison group of similar students who do not attend, or at minimum compare participants with different attendance doses. Fourth, train staff on consistent data collection and assign one person to manage missing data. Fifth, collect qualitative data from teachers and families to explain results. The robotics idea should be deferred or piloted separately; adding it now would dilute the reading focus and make attribution harder. The board should approve an evaluation plan with a timeline: baseline in the first month, mid-year attendance review, post-test at the end, and a report that separates findings from interpretation. This plan is credible because it fits the program's capacity and addresses the specific weaknesses in the current data.
3. How should the program director communicate with the foundation and the board about renewal?
Read the worked answer
The director should communicate early, not wait for the year-end report. A short mid-year update should state the attendance challenge, the missing data issue, and the corrective actions already underway. The report should distinguish outputs (students served, attendance) from outcomes (reading improvement) and explicitly note that the 64% figure is preliminary and not causal. The foundation is more likely to respect a credible improvement plan than a sudden claim of proof. The director should ask whether the foundation will accept a revised target or a learning report if attendance remains below 75%, and should propose a realistic evaluation design for the remaining months. The board should receive the same facts and approve the message. If the board wants to add robotics, it should treat that as a separate pilot with its own attendance and outcome measures, not as a substitute for reading results. The reasoning is that renewal depends on trust and learning, not on presenting weak data as strong evidence.