The Split-Night Sleep Trial
A school research team wants to know whether a 20-minute afternoon nap improves memory for vocabulary. They recruit 40 students and assign the first 20 volunteers to nap and the remaining 20 to stay awake and read. All participants study the same 30-word list in the morning. At 3 p.m., both groups take the same recall test. The nap group averages 22 words; the awake group averages 18. The team concludes that naps cause a 4-word gain and submits the report. A reviewer notes that volunteers were assigned in the order they signed up and that the nap group happened to include mostly students from an advanced language class. The team has two weeks to revise, no extra funding, and cannot recruit new participants.
1. Identify the main threat to internal validity and explain how it could produce the observed 4-word difference.
Read the worked answer
The main threat is selection bias, worsened by non-random assignment. Volunteers were assigned by sign-up order, and the nap group was dominated by advanced language students. Those students likely differ in baseline vocabulary and memory skill, so the 4-word gap may reflect group composition rather than the nap.
Because assignment was not random, the two groups are not comparable at the start. The study lacks a pretest to check equivalence, so the difference cannot be attributed to the treatment. A plausible rival explanation: advanced students already recall more words, and the nap had little or no effect.
2. Using only the existing 40 participants, propose a revised design that reduces this threat, and state one limitation that would remain.
Read the worked answer
A feasible revision is a within-subjects crossover: give every participant a pretest, then randomly split the 40 into two halves. Half nap first and stay awake second; the other half does the reverse on a second testing day, with different matched word lists and counterbalanced order. Comparing each student's nap versus awake score removes between-person differences such as class placement.
Limitation: order and practice effects remain, since students take two tests. Counterbalancing spreads but does not eliminate them, and two weeks may be too short to prevent remembering words across sessions. Also, the sample is still one school, so generalization stays limited.
3. The team cannot add participants. Describe one analysis they could run to test the selection explanation, and what result would weaken their conclusion.
Read the worked answer
They could collect each participant's prior language-course grade or a quick baseline vocabulary score and use it as a covariate in a regression or compare gains adjusted for baseline. If the nap group's advantage shrinks to near zero after controlling for baseline ability, the selection explanation gains support and the causal claim weakens.
A second option is to compare only students matched on baseline score (a post-hoc matched subsample). If the 4-word gap disappears within matched pairs, the original conclusion is undermined. Either analysis is weaker than random assignment, so results should be reported as suggestive, not causal.