The Misleading Statistic
A newspaper article states: 'Crime in the capital has risen 40% this year, proving that policing has failed.' A reader notices the article gives no absolute numbers and no comparison period. She finds, on checking, that the police recorded 50 incidents last year and 70 this year, and that a new reporting hotline was launched this year. She also notes the article compares the capital to a rural province with a much smaller population. A friend tells her: 'The percentage is a fact, so the conclusion must be true.'
1. Evaluate the argument's reasoning. Identify the specific logical and statistical errors.
Read the worked answer
The percentage is arithmetically correct but the inference is not.
- Base-rate neglect: 50 → 70 is a rise of 20 incidents. A 40% rise on a small base sounds dramatic but is a small absolute change. The article hides this.
- Reporting bias / confounder: the new hotline plausibly increased recorded incidents without any change in actual crime. 'Recorded crime' ≠ 'crime'. This alone can explain the rise.
- Invalid comparison: comparing the capital to a sparsely populated rural province ignores population size. Rates per 100,000 are needed, not raw counts.
- Non sequitur: even a genuine rise does not prove 'policing has failed'. Other causes (economy, demographics, reporting) are possible; 'proves' is too strong.
Verdict: the friend confuses a correct calculation with a valid argument. A fact can be true and the conclusion still unsupported.
2. What minimum information would make the claim testable, and how should the reader state her conclusion?
Read the worked answer
Minimum information needed:
- Absolute counts for both years (she has them: 50 and 70).
- Population figures for the capital and the comparison province, to compute rates per 100,000.
- A stable definition and reporting method across both years - did the hotline change what gets recorded?
- A comparable period and, ideally, several years to distinguish trend from noise.
A disciplined conclusion: 'Recorded incidents in the capital rose from 50 to 70. Because a new reporting hotline began this year, part or all of the increase may reflect better reporting rather than more crime. The claim that policing has failed is not supported by this data alone.'
Judgement point: the reader neither dismisses the statistic nor accepts the conclusion. She states what the data shows, names the confounder, and withholds the causal claim until it is testable.