The Flipped-Classroom Pilot
A college piloted a flipped classroom in one section of Introductory Statistics. Students chose whether to enrol in the pilot, so the 28 students in the flipped section were volunteers. The comparison section had 31 students and was taught by a veteran instructor at 11 a.m.; the pilot was taught by a new instructor at 8 a.m. At the end of term the flipped section's mean score was 78, against 72 in the comparison section, and 64% of the pilot students said they preferred the format. The dean says the results prove that flipping improves learning and wants it mandated for all sections. The programme coordinator notes that 9 of the 28 pilot students had previously taken a statistics course, compared with 3 of the 31 comparison students, and that the pilot section had a higher withdrawal rate (4 students vs 1).
1. Identify two rival explanations for the 6-point mean difference and explain how each could produce the gap even if flipping had no effect.
Read the worked answer
Rival explanation 1: selection bias through prior coursework. Nine of 28 pilot students (about 32%) had previously taken a statistics course, compared with 3 of 31 (about 10%) in the comparison section. Prior exposure can raise end-of-term scores independently of teaching method. If the pilot volunteers were more quantitatively prepared or more motivated, their mean could be 6 points higher even if the flipped method added nothing. The self-selection into the pilot makes this especially plausible: volunteers for a new format are not a random sample.
Rival explanation 2: differential attrition. The pilot lost 4 students (about 14% of 28) while the comparison lost 1 (about 3% of 31). If the students who withdrew from the pilot were weaker or less engaged, the remaining pilot group is a survivor group with a higher expected mean. The reported mean of 78 may describe only the stronger survivors, while the comparison mean of 72 includes more of its weaker students. This can manufacture a gap without any teaching effect.
Other credible rivals include the instructor confound (new vs veteran), the 8 a.m. vs 11 a.m. time slot, and the small samples. Any one of these could account for part or all of the 6-point difference.
2. What evidence would most strengthen the dean's causal claim? Describe a design or data source, and state what result would support the claim.
Read the worked answer
The strongest practical evidence would come from a randomised controlled trial with a common instructor and common time slot, or, if randomisation is impossible, a matched comparison with pre-test controls.
Design: randomly assign consenting students within the same course to flipped or traditional sections; hold the instructor, meeting time, assessment and syllabus constant; measure a pre-test at the start and the same post-test at the end; record withdrawal and attendance; and analyse by intention-to-treat so that attrition cannot silently reshape the groups.
Supporting result: the flipped group shows a significantly higher gain score (post-test minus pre-test) than the traditional group, with similar attrition and comparable prior preparation. If the raw post-test gap shrinks to near zero after controlling for prior coursework and attrition, the causal claim is weakened. A smaller but well-controlled gain is more persuasive than a larger uncontrolled gap.
The preference survey is not a substitute: it measures satisfaction among volunteers, not learning. The dean's claim needs an outcome measure and a design that rules out selection, instructor and attrition effects.
3. Does the 64% preference figure justify mandating the format for all sections? Explain what inference it supports and what it does not support.
Read the worked answer
It supports a modest descriptive inference: among the 28 students who volunteered for the pilot, a majority-about 18 if 64% is exact-said they preferred the flipped format. It does not support a mandate for all sections.
What it does not establish:
- Learning effectiveness. Preference is not achievement. Students can prefer a format that teaches them less, or dislike a format that teaches them more.
- Generalisability. The respondents were volunteers who chose the pilot. Students who avoided it may have different preferences and may respond differently to a mandate.
- Causal direction. The 64% may reflect novelty, the new instructor, the 8 a.m. slot, or the chance to self-select into a preferred format.
- Implementation capacity. A mandate requires trained instructors, pre-class materials and classroom space. A pilot with one willing instructor says little about scaling.
A defensible conclusion is that the pilot is promising enough to justify a larger, better-controlled trial-not that the college should mandate the format. The dean is moving from 'a majority of volunteers liked it' to 'everyone should be required to do it', which is an overreach.