The Digital Archive Trap
A student's History Project examines how a 1980s public health campaign was reported in newspapers. She has assembled 60 digitised articles from an online archive using keyword searches. She plans to argue that coverage 'shifted from panic to reassurance' over 18 months. Her teacher asks how she selected the 60 articles. She admits she searched for terms like 'crisis,' 'scare,' and 'reassurance,' and kept the results that seemed relevant. She has 4,000 words and three weeks.
1. What is wrong with the student's sampling method, and how does it undermine her thesis?
Read the worked answer
Her method is circular: she searched for the emotional vocabulary her thesis is about, then concluded that the vocabulary changed over time.
- By searching 'crisis' and 'scare,' she guarantees that early panic language appears in her sample. By searching 'reassurance,' she guarantees later reassurance language. The sample cannot test the claim because it was built from the claim.
- She has no denominator. Sixty articles is meaningless without knowing how many articles the newspapers published on the topic in total. A shift in the proportion of coverage is a real finding; a shift in what her keyword search returned is an artefact.
- Selection on relevance adds a second bias: she kept what 'seemed relevant,' which likely means what fit her expected narrative.
The fix is a defined corpus: choose a set of newspapers, a date range, and a sampling rule (e.g. every article mentioning the campaign in the health pages of two papers, sampled weekly). Then code all of them for tone, whether or not they mention crisis or reassurance. The thesis should be revised to a question the corpus can answer - for example, whether the proportion of alarmist framing declined - and she must be willing to report a null or contrary result.
2. With three weeks left, how should she restructure her project to be feasible and defensible?
Read the worked answer
She should trade breadth for rigour. A smaller, systematically chosen corpus is far stronger than 60 cherry-picked articles.
Week 1 - Build the corpus. Pick two newspapers with good digital coverage. Define a date window (say 18 months) and a sampling rule: every article in the health section mentioning the campaign, or one issue per week. Record the total number found - this is her denominator. Aim for 30–50 articles selected by rule, not by relevance.
Week 2 - Code and analyse. Create a simple coding scheme: framing (alarm / reassurance / neutral / mixed), source quoted (official, doctor, public), and placement (front page, health page). Code every article, including dull ones. Tabulate by month to see whether proportions shift.
Week 3 - Write and reflect. Structure: 600 words on method and sampling, 500 on context, 2,000 on findings with a table, 500 on limitations, 400 on conclusion. The limitations section is essential: she should state that two newspapers are not 'the press,' that digitised archives over-represent some titles, and that her coding involves judgement.
She should also pre-commit to reporting what she finds. If coverage did not shift, that is a finding, and arguing why the expected shift did not appear is a stronger project than confirming her original hunch.