The Library-Door Questionnaire on Night-time Mobile Use
Two Class 12 students in a Dibrugarh school want to find out whether late-night mobile use lowers examination marks. They stand outside the school library on three afternoons and hand a questionnaire to every student who walks out, collecting 84 responses from a class of 320. The questionnaire asks: 'How many hours do you use your phone at night?' and 'Do you think your marks have fallen because of your phone?' They find that students reporting more than three hours of night use also report lower marks, and conclude that mobile use causes lower marks. Only nine respondents are from the science section, which has the school's highest marks. The library is used mainly by students who stay back for extra classes, and the survey is done in the week before the pre-board examination. The teacher is impressed but asks whether the result would hold for all 320 students, whether self-reports can be trusted, and whether the students needed anyone's permission before questioning 16- and 17-year-olds.
1. Identify the sampling and measurement problems in this study and explain, for each, the direction in which it could distort the correlation.
Read the worked answer
Four problems, each with a likely direction of bias:
- Convenience, self-selected sample. Only students who walk out of the library on three afternoons are included. Library users are likely to be more studious and to stay for extra classes; they are not representative of the 320. The difficulty is not mainly the size of 84 against 320 - it is that selection is not random.
- Restricted range on the marks variable. Only nine respondents are from the science section, which has the school's highest marks. Under-sampling the top of the marks distribution truncates the range and can weaken or distort the observed correlation.
- Timing confound. The survey runs in pre-board week, when stress and revision raise late-night phone use (as a break) and also disturb sleep and marks. The correlation may be produced by the exam week itself rather than by phones.
- Common-method and demand bias. Both variables come from the same self-report at the same time, and the second question ('Do you think your marks have fallen because of your phone?') supplies the expected answer, inviting agreement. This inflates the association.
Remedy: draw a random sample from the class register (for example every fourth name), survey in a neutral week, and take marks from school records rather than from self-report, so the two variables are measured independently.
2. Why can the students not conclude that mobile use causes lower marks? Name three alternative explanations and describe one design change that would address at least two of them.
Read the worked answer
A correlation between two variables does not show that one causes the other, because at least three alternatives survive the data:
- Reverse causation. Poor marks may lead to late-night phone use: a student anxious about failing scrolls at night to avoid studying. The observed relation fits this direction just as well.
- A third variable. Sleep duration, home study space, parental supervision or prior achievement could raise both night-time phone use and poor marks. If poor sleepers both use phones at night and perform badly, the phone–marks link may be entirely due to sleep.
- Selection. Students who are already disengaged may both use phones at night and score low, so a third factor produces both.
Design change: a prospective (longitudinal) design - measure phone use in a neutral month, then take the next examination's marks, statistically controlling for prior marks and sleep. Fixing the time order removes reverse causation, and measuring the covariates addresses the third-variable problem. An alternative is a matched-groups comparison in which students with similar prior marks and sleep habits differ in night-time phone use.
Even after this, the students should phrase the finding as 'associated with', because a causal claim also needs a plausible mechanism - for example, that shortened sleep impairs consolidation of what was studied.
3. List the ethical requirements these students must satisfy, keeping in mind that participants are 16–17-year-old school students.
Read the worked answer
Points the students must address before collecting any data:
- Consent for minors. At 16–17 the participants cannot give legally valid consent alone. The researchers need permission from a parent or guardian plus the assent of the student. A signature collected from a captive class is not enough.
- Institutional permission. The principal's approval is needed before questionnaires are handed out on school premises.
- Voluntariness and no coercion. Students must be free to refuse or withdraw without penalty. If a teacher collects or chases the forms, refusal becomes difficult, so the student researchers should collect them themselves, using a sealed drop box.
- Anonymity and confidentiality. No names, roll numbers or handwriting that makes a response traceable; results reported only as group totals.
- No harm and debriefing. Asking 'do you think your marks have fallen because of your phone?' can heighten anxiety in pre-board week. The researchers should explain the purpose afterwards, share the group result, and explain what a correlation can and cannot show.
- Accurate reporting. They must not publish 'phones cause lower marks' in the school magazine, since the design cannot support that statement; the wording should be 'students who reported more night-time use also reported lower marks'.
Running through all six is one principle: the participants' welfare and autonomy take priority over the tidiness of the result.