The Questionnaire That Measured Two Things at Once
Marcus is investigating whether students who report higher 'academic pressure' also report poorer sleep quality. He writes a 12-item online survey. Items 1–4 ask about pressure (e.g. 'I feel I must not disappoint my family', 'I worry about ranking'), items 5–8 ask about sleep ('I take a long time to fall asleep', 'I wake during the night'), and items 9–12 ask about general anxiety ('I often feel tense', 'I worry about many things'). All items use the same 1–5 agreement scale and are presented in one block, pressure first. He surveys 95 Year 11 and 12 students during a free period, achieving a 61% response rate. His correlation between the pressure block and the sleep block is r = 0.48, and he concludes that academic pressure causes poor sleep. A classmate points out that the anxiety items also correlate strongly with both blocks.
1. Explain how the anxiety items threaten Marcus's interpretation, and distinguish this from the separate problem of causal language.
Read the worked answer
Confounding by a third variable. The anxiety block correlates with both pressure and sleep. A plausible model is that general anxiety drives both the tendency to endorse pressure items and the tendency to report poor sleep, producing a spurious pressure–sleep association. Marcus cannot separate 'academic pressure' from 'general anxious disposition' with this instrument because the two are measured by overlapping, similarly worded items in the same block.
Causal language is a separate error. Even if the association were genuine and unconfounded, a cross-sectional correlation at one time point cannot establish that pressure causes poor sleep; the direction could reverse (poor sleep makes students feel less able to cope, which they report as pressure). These are two distinct problems: one about what is being measured (construct confounding), the other about what can be inferred from the design (causation from correlation). Marcus needs to fix both, and fixing one does not fix the other.
2. Recommend specific changes to the instrument and analysis that would let Marcus test his question more cleanly.
Read the worked answer
Instrument. Separate the constructs: keep pressure items focused on externally imposed demands (deadlines, family expectations, ranking) and remove or relocate the anxiety items to a distinct, clearly labelled section. Randomise item order or at minimum split blocks so pressure does not always precede sleep. Pilot the revised scale with a small group and check internal consistency within each block; if pressure and anxiety items load on one factor, the scale is measuring one thing and must be revised. Include a validated sleep measure rather than four ad hoc items.
Analysis. Report the correlation between pressure and sleep controlling for the anxiety score (partial correlation or regression with anxiety as a covariate), and report the zero-order and partial coefficients side by side so the attenuation is visible. Check whether pressure and anxiety are so collinear that the model cannot separate them; if so, say so rather than reporting a misleading coefficient.
Response rate. 61% is a moderate response rate and non-responders may differ systematically; Marcus should report it as a limitation and, if possible, compare responders and non-responders on available characteristics.
3. Marcus's teacher says the study is still worth reporting even if the confounding cannot be eliminated. Justify this position and state the precise claim Marcus may make.
Read the worked answer
The study remains worth reporting because it can establish whether an association exists and whether it survives adjustment for anxiety - a genuinely informative result either way. If the partial correlation drops sharply, that is evidence that the pressure–sleep link is largely explained by general anxiety, which is a finding, not a failure. If it persists, that motivates a stronger design.
Permissible claim. 'Among 58 responding Year 11–12 students, self-reported academic pressure was associated with poorer self-reported sleep (r = 0.48); after adjusting for general anxiety the association was [value], indicating that [X] of the association is attributable to anxious disposition. This is cross-sectional and cannot establish direction or causation.' He must not say pressure causes poor sleep, must report the response rate and the single-time-point design, and should frame the result as motivating a longitudinal or diary study.