The Subgroup Behind the Average
At the January data review, your 640-student K-8 school looks like a success: overall reading proficiency rose from 56% to 62%. Disaggregation tells a different story. Grade 3 students with disabilities (n=38) fell nine points, and Black boys in grades 4-5 (n=27) fell seven points. Several teachers say the screened cohort 'just had more needs this year,' and one suggests the intervention teacher is being 'drained' by behavior problems. Your data team reviews only grade-level averages and has never examined a subgroup across two time points. The reading coach proposes a 30-minute intervention block, but the only open schedule slot would displace science in grades 4-5. The district will publish school-level dashboard data in three weeks, and your supervisor has asked you to 'lead with the gains' at the February board meeting. You have one scheduled staff meeting before then and a data team of six teachers.
1. Before changing the schedule, what analyses and validity checks would you require? What would make you treat the subgroup drops as a real signal rather than noise?
Read the worked answer
Before changing schedules, test whether the pattern is real and what it means. With n=38 and n=27, a nine-point drop is roughly three or four students; one family move or one misadministered screening can produce it. Check denominators and mobility: are these the same students as in the fall, or a different cohort? Check administration fidelity, accommodations used during screening, and whether cut scores changed. Then disaggregate within the subgroup by classroom, teacher, attendance, and English learner status; a subgroup average can hide two very different classrooms. Look for Simpson's paradox: the overall gain may be driven by grades 6-8 while grade 3 fell. Run a fall-to-winter matched analysis for students with two data points, and examine item-level error patterns, not just the composite. Also examine referral and identification practices: if students with disabilities were newly identified or moved in, the composition changed.
Ask what the data cannot tell you: whether instruction changed, whether the intervention group is overrepresented, and whether students are receiving the services on their IEPs. Check implementation logs before attributing cause. Set a rule in advance for what counts as a real signal, for example the same pattern at the next benchmark plus classroom-level confirmation, so the team does not chase noise or dismiss a real harm. Suppress small cells in public displays and keep student-level data inside the inquiry team, not in the board packet.
2. Design the staff data-inquiry response. How do you avoid deficit framing, and how do you handle the schedule tradeoff between intervention and science?
Read the worked answer
Design the response as a Tier 1 inquiry, not a deficit intervention aimed at 'those kids.' Start the staff meeting with the pattern and a norm: we will examine what we taught, how, and under what conditions before we describe students. Use three teacher-led groups for grade 3, grades 4-5, and the intervention and schedule question. Each group uses a 20-minute protocol: what do we see, what questions does this raise, what is one change we can test in three weeks? Include the intervention teacher and a special educator; their absence would guarantee an incomplete explanation. Require each group to bring evidence of implementation, not just opinions.
On the schedule, do not accept the first binary. The 30-minute block may be found in a low-value use of time such as homeroom, study hall, or a repeated advisory, or through push-in support during science that uses disciplinary literacy rather than removing science. If a tradeoff is unavoidable, run a six-week trial with explicit exit criteria and protect lab time. Measure both reading and science outcomes so the school can see the cost. Avoid a permanent schedule change based on one screening. Report back at the next data team meeting with student-level progress monitoring, and keep the focus on whether the change improved outcomes for the named students.
3. What do you say to the board and public in three weeks, given incomplete data and your supervisor's request to lead with the gains?
Read the worked answer
Lead with the gains and the gap in the same breath. Saying only that proficiency is up six points would be true and misleading; the board and public will eventually see the disaggregated dashboard, and the credibility cost of spin is higher than temporary comfort. A workable frame: 'Overall reading is improving, which shows our instruction can move outcomes. Two groups did not share that gain, and we are treating that as an instructional problem, not a student problem. Here is what we are checking and when we will report back.' Give the numbers with sample sizes and a caution that one benchmark is a signal, not a verdict. Do not name students or small cells, and do not speculate about causes you have not tested.
Prepare for the question about why those students are behind with a systems answer: attendance, access to grade-level content, intervention fidelity, identification practices, and teacher assignment. Avoid language that locates the problem in families or motivation. State the next public checkpoint: date, measure, and owner. Commit to reporting whether the gap narrowed, not just whether the average rose. If your supervisor wants a gains-only message, say privately that you will lead with gains but cannot omit the subgroup data, and offer to co-present so the framing is shared. The ethical test is whether you would defend the message if the subgroup data appeared on the front page.