A 92% accurate test that is wrong 96 times out of 100
A district hospital is deciding how to use a new rapid antigen test. In a validation study the test was run on 2,000 people whose true status had already been confirmed by a laboratory method: of the 1,000 infected people, 920 tested positive; of the 1,000 uninfected people, 960 tested negative. The test is cheap and gives a result in 15 minutes, so the hospital plans to deploy it in two settings - a general outpatient clinic where the disease is present in about 1 person in 500, and a specialist referral clinic where about 1 person in 20 has the disease. At a briefing a doctor says: "The test is 92% accurate, so a positive result means a 92% chance that the patient has the disease." A journalist challenges this claim. The board asks for the actual predictive values in both settings before approving the policy, and also whether a second independent test on the same patient would settle the matter.
1. Compute the sensitivity and specificity from the validation study, then the positive predictive value in the general outpatient clinic. Is the doctor's 92% claim defensible?
Read the worked answer
Sensitivity = 920/1000 = 0.92; specificity = 960/1000 = 0.96, so the false-positive rate is 0.04.
Take 10,000 people from the general clinic (prevalence 0.002):
- Diseased: 20 → positives = 0.92 × 20 = 18.4
- Healthy: 9,980 → positives = 0.04 × 9,980 = 399.2
PPV = 18.4/(18.4 + 399.2) = 18.4/417.6 = 0.044, i.e. about 4.4%.
By Bayes' formula: (0.92 × 0.002)/[(0.92 × 0.002) + (0.04 × 0.998)] = 0.00184/0.04176 = 0.044.
The claim is indefensible. It confuses P(positive | disease) = 0.92 with P(disease | positive) = 0.044. In this clinic more than 95 of every 100 positives are false positives.
2. Compute the positive predictive value in the specialist referral clinic and explain why the same test gives a different answer.
Read the worked answer
Prevalence here is 1/20 = 0.05.
PPV = (0.92 × 0.05)/[(0.92 × 0.05) + (0.04 × 0.95)] = 0.046/(0.046 + 0.038) = 0.046/0.084 = 0.548, about 55%.
Same kit, same sensitivity, same specificity - only the pre-test probability changed. This is the key distinction: sensitivity and specificity are properties of the test (they describe how it behaves among the diseased and the healthy), whereas the predictive value is a property of the test plus the population it is applied to. As prevalence rises, the healthy group shrinks and false positives become a smaller fraction of all positives.
Board decision: a single positive result is informative enough to act on in the referral clinic (about 55%) but not in the general clinic (about 4%), where it should trigger confirmatory testing rather than treatment or reassurance.
3. A patient in the general clinic tests positive twice on independent runs. Compute the new PPV, and decide whether that justifies treating the patient when the treatment seriously harms 1 in 100 recipients.
Read the worked answer
Both positive, general clinic:
P = (0.92)² × 0.002 / [(0.92)² × 0.002 + (0.04)² × 0.998] = 0.001693/(0.001693 + 0.001597) = 0.001693/0.003290 = 0.515, about 51%.
The PPV rises from 4.4% to 51.5% - roughly eleven-fold - because the false-positive probability is squared while the true-positive probability falls only slightly.
Decision: with one positive, about 96 of every 100 treated patients are healthy, and roughly 1 in 100 of those treated is seriously harmed - a bad trade against about 4 who benefit. With two positives, roughly half are truly infected, so treatment is defensible.
One caveat must be stated: the calculation assumes the two runs are independent. Same kit, same operator error and the same cross-reacting condition break that assumption, and the true PPV would then be lower. So the correct sequence is test → confirm by a different method → treat.