The Public Dashboard That Overstates Its Confidence
A city health department publishes a public dashboard showing neighborhood-level rates of a chronic condition. The dashboard uses small-area estimation because some neighborhoods have fewer than 50 cases. The estimates carry wide confidence intervals - for 9 of 40 neighborhoods the interval spans a two-fold range - but the map displays only point estimates in five color bands. A local journalist downloads the data, ranks neighborhoods, and publishes 'worst neighborhood in the city' based on the top band. That neighborhood is predominantly low-income and has a strong community organization that is now demanding resources. The department's epidemiologist warns that the ranking is unstable and would likely change with next year's data. The communications team wants to keep the map simple because 'the public won't understand confidence intervals.'
1. Identify the ethical problems with displaying only point estimates, and who is harmed.
Read the worked answer
The core problem is false precision: the map presents estimates as if they were measurements. Five color bands imply a certainty the underlying data does not support, and for 9 of 40 neighborhoods the true rate could plausibly be half or double the displayed value.
Harms:
- The named neighborhood is stigmatized by a 'worst in the city' label that the data cannot support. Residents may face property-value effects, insurance consequences, or reputational harm.
- The community organization is put in the position of advocating on a shaky number. If next year's estimate drops, its credibility suffers even though nothing changed.
- Other neighborhoods may be under-resourced because they landed in a lower band by noise.
- The public and the journalist are misled about what the data says, which undermines trust when the instability becomes visible.
The communications team's concern is real but misdirected: the fix is not to hide uncertainty but to present it so it cannot be misread. Suppressing uncertainty to avoid confusion is a transparency failure that shifts the cost onto the people the map is about.
2. Propose a redesign that preserves usability and honesty, and address the 'public won't understand' objection.
Read the worked answer
A workable redesign:
- Show uncertainty, don't bury it. Use a graded or hatched treatment for estimates whose interval crosses a band boundary, or display intervals in the tooltip and a companion table. A common, tested pattern is a 'reliability' overlay - solid color where the estimate is stable, faded where it is not.
- Stop ranking. Remove sortable league tables and add a plain-language note: 'Neighborhoods are not ranked; small differences are within the margin of error.'
- Lead with the interval in the legend. 'Estimated rate: 12% (plausible range 7–21%)' is understandable. People read weather forecasts with ranges every day.
- Add a one-paragraph methods note in plain language and a downloadable technical appendix.
- Pre-brief the journalist and the community organization before publication, with the interval data and an offer to walk through it.
On the objection: the evidence from risk communication is that people handle uncertainty better than they handle being misled. The failure mode is not confusion at the legend; it is a confident wrong ranking that damages a neighborhood and then collapses. Simplicity is a design goal, but it cannot come at the cost of a claim the data does not support. If a simpler display is needed, simplify the number of bands and show ranges - do not delete the ranges.