Keyword Hits and the 1951 Waterfront Dispute
A researcher is using a newly digitised archive of 40,000 newspaper clippings about the 1951 waterfront dispute. The archive is searchable by keyword. A search for 'communist' returns 1,200 hits: 900 from two conservative metropolitan dailies, 200 from a labour-affiliated weekly, and 100 from regional papers. The labour weekly's OCR is poor; 'communist' often appears as 'comniunist'. Only 12,000 of the 40,000 clippings are digitised; the rest are indexed only by date and newspaper title. The researcher has six weeks and concludes that 'the dispute was overwhelmingly portrayed as a communist conspiracy'. A supervisor asks for a methodological justification before the claim can be used in an essay.
1. Evaluate the researcher's conclusion. What inferential errors are involved?
Read the worked answer
The conclusion moves too quickly from a search result to a claim about public portrayal. The first problem is the denominator: 1,200 hits is meaningless without knowing how many total clippings were searched, how many were relevant, and how often alternative framings appeared. The second is digitisation bias: only 12,000 of 40,000 clippings are digitised, and the non-digitised majority may differ systematically by newspaper, region or date. The third is OCR bias: the labour weekly's poor text means its use of 'communist' is undercounted, so the search itself is skewed against the labour perspective. The fourth is source concentration: 900 of the 1,200 hits come from two conservative metropolitan dailies, which may reflect editorial coordination or simply their greater output, not a society-wide consensus.
The researcher also conflates frequency with dominance and mentions with meaning. A hit shows that a word appeared, not that it was endorsed, criticised or quoted. 'Overwhelmingly portrayed' requires evidence of tone, placement and repetition across a representative sample. A defensible conclusion would be narrower: 'In the digitised conservative metropolitan press, communist framing was frequent.' That claim would still need qualification about the missing papers.
2. Design a feasible sampling strategy for the six-week timeframe that would allow a more defensible claim.
Read the worked answer
A feasible strategy would use stratified sampling rather than keyword totals. First, stratify the 40,000 clippings by newspaper type (conservative metropolitan, labour-affiliated, regional, religious or community) and by month across the dispute. Second, draw a random or systematic sample from each stratum, including clippings that are not digitised; the date-and-title index makes this possible, and microfilm can be read for selected dates. Third, read whole pages or articles rather than isolated keyword hits, and code each item for explicit communist framing, implied framing, criticism of such framing, and neutral industrial coverage. Fourth, run a control search for terms such as 'strike', 'lockout' and 'union' to estimate how much coverage was not about communism at all.
Within six weeks the researcher cannot read everything. The aim is therefore transparency, not exhaustiveness: record the sample size, the selection method and the limits. A claim such as 'in a sample of 300 clippings, conservative metropolitan papers framed the dispute as communist-led more often than labour or regional papers' would be defensible. The labour weekly's OCR problem should be handled by reading the original or microfilm for the sampled issues, not by trusting the keyword search.