Spacing, testing, and a claim of additivity from an underpowered 2×2
An educational psychology lab runs a two-experiment study on retrieval practice. In Experiment 1, 60 undergraduates learn 40 Swahili–English word pairs in one of two schedules - four study blocks in a single session, or four blocks spread across four days - crossed with activity type on each block (restudy the pairs, or take a recall test). All participants return after seven days for a final cued-recall test. Results: spaced beats massed (M = 71% vs 54%), testing beats restudying (M = 68% vs 57%), and the interaction is not significant (p = .31). In Experiment 2, participants judge immediately after learning how well they will remember the material in a week. Those in the massed condition give higher confidence ratings than those in the spaced condition, even though Experiment 1 shows the opposite for actual retention. Colleagues plan to submit Experiment 1 as evidence that "spacing and testing are additive" and Experiment 2 as evidence of "metacognitive illusion," and to recommend that students always space and always self-test.
1. Evaluate the claim that Experiment 1 demonstrates additivity. What design and analytic features would you check before accepting it?
Read the worked answer
A non-significant interaction is not evidence of additivity. The 2×2 design yields roughly 15 participants per cell. Tests of interaction are markedly less powerful than tests of main effects - as a rough rule they need about four times the sample of a main-effect test to detect an effect of the same size - so p = .31 is close to uninformative. Demonstrating additivity requires either an equivalence test with a pre-specified smallest effect size of interest, or a Bayes factor supporting the null, not a failed significance test.
Design checks. (1) Total study time and number of retrieval attempts: a test block contains a recall attempt plus feedback, so "testing" conditions may receive more time-on-task than restudy conditions unless equated. (2) Spacing is inherently confounded with retention interval since first exposure; both groups are tested seven days after their final session, which is the correct control, but the lag between sessions should be justified relative to the seven-day criterion. (3) Item counterbalancing so that pair difficulty cannot differ across cells. (4) Whether the massed group's four blocks were genuinely contiguous or merely same-day.
On the evidence given, the safer statement is that both manipulations produced sizeable main effects in the expected directions, with the joint effect unresolved.
2. Experiment 2 finds that massed learners are more confident. What is the metacognitive interpretation, and what alternative explanations should be ruled out before calling it an illusion?
Read the worked answer
The standard interpretation is a fluency-based metamemory illusion. Judgments of learning are heavily influenced by how easily material comes to mind at the moment of judgment. Under massed practice the items were just processed repeatedly, so retrieval feels smooth; that experienced fluency is misread as a prediction of future retrievability. A related mechanism is stability bias - people underestimate how much forgetting will occur - and the immediate timing of the judgment maximizes both biases.
Alternatives to rule out. (1) Demand characteristics: students may infer the experimenter expects massed practice to work, or may anchor the numeric scale differently across conditions. (2) Item or order effects: if the massed group saw items in a different order or received a different number of exposures before judging. (3) Differential judgment timing relative to the last study episode. (4) Selective attrition if confidence was collected after some participants dropped out.
A stronger design collects judgments both immediately and after a delay, includes a post-test judgment to show the dissociation directly, and manipulates fluency independently of actual learning (for example, perceptual clarity or study duration) - if fluency is the mediator, confidence should follow the manipulation while recall does not. As reported, the data are suggestive but the causal claim about metacognitive illusion rests on a single between-groups comparison.
3. The recommendation "always space and always self-test" is proposed for students. What boundary conditions should qualify it?
Read the worked answer
Spacing. The optimal gap scales with the retention interval: for a test a week away, gaps on the order of a day are reasonable; for an exam months away, gaps of weeks. Spacing that is too wide for the criterion behaves like massing. Spacing also assumes the learner re-encounters the material, which requires a schedule the student will actually follow.
Testing. The benefit is largest when the practice test format resembles the criterion task (transfer-appropriate processing): recall of definitions helps recall, but transfer to application problems needs practice with application. Tests also require feedback, particularly for errors, and they impose a cost: unsuccessful retrieval without feedback can entrench errors. Desirable difficulties are desirable only when the learner has enough background knowledge to succeed at retrieval; for low-knowledge learners, studying worked examples first is more efficient.
Individual and material factors. Complex, intercorrelated material (texts, conceptual systems) benefits less from simple flashcard testing than from elaborative strategies; high test anxiety can turn retrieval practice into an aversive experience unless the stakes are low. Finally, metacognitive calibration must be addressed: because massed study feels more effective, students will not adopt spacing on their own without explicit instruction and structured schedules.