Flat Screener Scores After Eight Weeks: Program Failure or Delivery Failure?
A district buys a three-year adaptive reading program and directs every K-5 teacher to use it 20 minutes daily, three days a week, as a station. After eight weeks, the mid-year screener shows 9 of 24 second graders in Ms. Barrow's class remain two or more grade levels below benchmark - the same count as in the fall. Ms. Barrow wants to drop the program. The technology integrator pulls usage logs: the class averaged 11 minutes per session, 6 students account for 58% of total minutes, and two of the lowest-performing students have logged in only four times. Ms. Barrow's station rotation during that block is teacher-led small-group reading, which she says she will not give up. The principal wants a decision at the next data meeting.
1. Is the flat screener evidence that the program failed? What would you check before deciding?
Read the worked answer
Not yet, and the logs are the reason. The implementation does not match the design: 11 minutes against a 20-minute target, 58% of minutes concentrated in six students, and two of the neediest students at four logins. With usage that uneven, a flat result is uninterpretable - it is equally consistent with an ineffective program and with an effective program that was barely delivered to the students who needed it. Before recommending continuation or cancellation, check three things: minutes per student against the vendor's fidelity threshold; whether entry placements were accurate, since misplacement produces weeks of too-easy content and no growth; and whether the program's content aligns with what the screener measures. Also ask whether eight weeks is long enough - adaptive programs typically need 10-12 weeks of consistent use before growth appears on a screener.
2. Ms. Barrow will not give up her teacher-led small group. Propose an implementation change that keeps it.
Read the worked answer
Do not trade away the small group; it is probably the strongest element in the room, and the program was never meant to replace it. Restructure the block instead. In a 60-minute period, run three 18-minute stations: adaptive program, independent reading/writing, and teacher-led group. The failure was logistical, not philosophical - two of the lowest students never logged in - so build a first-five-minutes routine in which students log in and the teacher visually confirms green status before stations start, and tape a printed login card inside each device tray. Fix device seating so no student shares, and have the teacher spend five minutes on the dashboard each Friday. If the schedule truly cannot hold three stations, cut sessions to 15 minutes but hold the daily frequency; for adaptive tools, consistency beats duration. Commit to six more weeks before judging.
3. How would you frame the decision at the data meeting so it does not become program-versus-teacher?
Read the worked answer
Frame it as an implementation question, and bring the numbers that make the distinction visible. Show fidelity first: average minutes per student, a simple bar chart contrasting the six heavy users with the rest, and the two students at four logins. Then show the outcome. Say plainly that the flat screener cannot yet be attributed to the program and that Ms. Barrow's objection is a legitimate reading of the data. Put the proposal on the table as a test: six weeks of verified daily use with a stated fidelity threshold, then the same screener. Agree in advance what result would support continuing, what would support changing programs, and who decides. Keep the principal's decision date, but convert the question from 'keep or drop' into 'what do we fix first, and how will we know.'