Tutoring Article: When 'Proven' Outruns the Design
A local newspaper reports on an 18-month Saturday tutoring pilot. The article begins, 'The results are in: Saturday tutoring has proven to work and should be replicated in every elementary school.' It reports that 240 third graders who attended at least 15 sessions gained an average of 6.2 points on the district reading test, while students who attended fewer than five sessions gained 2.1 points. The article quotes the program director saying, 'This is the model every school needs.' Later paragraphs note that 80% of the pilot students were recruited from two schools already receiving additional reading support, and that 30% of sessions were led by uncertified volunteers. The article does not report baseline reading scores, student selection methods, or whether the same test form was used. A parent asks you to evaluate whether the article's conclusion is trustworthy.
1. Is the phrase 'proven to work' justified? Identify the strongest threat to the article's causal claim and explain why it matters.
Read the worked answer
No. The strongest threat is selection bias: students were grouped by attendance, not assigned. Students who attended at least 15 sessions may already have been more motivated, better supported, or stronger readers than students who attended fewer than five. The article also notes that 80% of pilot students came from two schools already receiving extra reading support, so school-level resources could explain the gain. Missing baseline scores make it impossible to know whether the high-attendance group started ahead or was already improving faster. The uncertified volunteers may affect quality but are a secondary issue. 'Proven' would require a controlled comparison and pre-specified outcomes; this design supports at most a promising association.
2. What additional evidence would most help a school decide whether to adopt the model? Give two items and explain how each would change the interpretation.
Read the worked answer
Two high-value items are baseline reading scores and a matched or randomized comparison group. Baseline scores would show whether the 6.2-point gain reflects growth or simply a higher starting point; if high attenders began well above the comparison group, the gain is less impressive. A matched or randomized comparison would address self-selection and the extra-support schools by comparing similar students who did and did not receive tutoring. A third useful item would be implementation records showing whether sessions used a consistent curriculum and certified versus volunteer tutors, because that affects whether results can be replicated. Without these, a school should pilot the model with clear comparison data rather than adopt it districtwide.
3. Revise the article's opening claim into a headline that is accurate and not misleading.
Read the worked answer
Headline: 'Saturday Tutoring Pilot Shows Promise, but Study Design Cannot Prove It Works.'
The revision keeps the positive signal, since high attenders gained more, but removes 'proven' and 'every school.' It signals the key limitation without dismissing the program. A headline should not imply causation when the comparison groups were self-selected and schools differed in prior support. If space allows, the subhead should name the 6.2-point versus 2.1-point gap and note that students were not randomly assigned.