NBDHE study materials
NBDHE study resources

Did Stopping Fluoridation Cause the Caries Increase? A Press Release Waiting to Happen

NBDHE · Put your knowledge to work in context.

All case studies

Did Stopping Fluoridation Cause the Caries Increase? A Press Release Waiting to Happen

In 2016 a city of 85,000 stopped fluoridating its water after 40 years. In 2021 the public health dental clinic reported that mean dmft among kindergarten children had risen from 1.8 to 2.6, and the board is preparing a press release attributing the increase to the loss of fluoridation. You are asked to review the data first. Within the same period: a new pediatric dental clinic opened in 2018 serving Medicaid families; Medicaid eligibility expanded in 2019; school screening moved from a school-based visual exam by two calibrated hygienists to a clinic-based exam by a rotating staff of five dentists; screening participation rose from 55% to 78%; and about 2,000 immigrant families moved into the city. The state contains other comparably sized cities, about half of which still fluoridate.

1. Identify the threats to internal validity in this comparison, and state the likely direction of bias for each.

Read the worked answer

The design is a single-group pre-post comparison with no control - the weakest design for a causal claim. The threats, and their likely direction:

  1. Instrumentation / examiner change (the largest threat). Two calibrated hygienists doing school-based visual exams became five rotating dentists doing clinic-based exams. Different examiners, different lighting, different criteria interpretation, and no inter-rater reliability data. A shift in measured dmft of this size can be produced by examiner drift alone. Direction: unknown, but potentially all of the 0.8.
  2. Selection / participation bias. Participation rose from 55% to 78%. The children added are disproportionately those who previously did not attend school screening - often the children with the highest need and least access. A rising mean dmft can therefore reflect who got counted, not who got caries. Direction: inflates the increase.
  3. Secular trend / co-intervening programs. A new pediatric Medicaid clinic and Medicaid expansion both increase access to diagnosis and treatment. More diagnosed, more treated-and-counted surfaces (dmft counts filled surfaces, not just decayed). Direction: inflates dmft, and inflates it more after 2018–19 than before.
  4. Population change / migration. Roughly 2,000 newcomer families, plausibly with different caries experience, fluoride exposure history, and language/access barriers. Composition change can move a mean without anyone's caries changing. Direction: unclear, must be measured.
  5. Maturation / time lag. Caries takes years to manifest; 2016 to 2021 spans the right window, so timing is consistent - but timing consistency is not evidence and should not be presented as though it were.
  6. Ecological inference. Group means cannot establish individual exposure-outcome links; nothing here measures whether these specific children drank fluoridated water.

What is not a threat: the biological plausibility of fluoridation's effect. The mechanism is well established; the problem is entirely with this study's ability to attribute this change to that cause.

2. Propose a stronger design that could actually answer the question, including what to measure and how to analyze it.

Read the worked answer

Design: difference-in-differences using neighboring cities as controls.

  • Exposure group: the city that stopped (post-2016 non-fluoridated). Comparison group: two to four comparably sized state cities that retained fluoridation, matched on median income, race/ethnicity composition, Medicaid participation, and baseline kindergarten dmft.
  • Outcome: mean dmft (and ideally also the prevalence of untreated dentinal caries, using ICDAS or a validated visual index) among kindergarten children, measured at the same age and same time of year.
  • Key analysis: compare the change from baseline to follow-up in the exposed city with the change over the same interval in the comparison cities - the difference of differences. If caries rose 0.8 in the exposed city and 0.2 in the comparison cities, the estimate of effect is 0.6, not 0.8. Report a confidence interval.

Fix the measurement before you fix the analysis

  1. Same examiners, same criteria, same setting, both periods or both groups. Use two calibrated examiners with an inter-rater kappa target (e.g., ≥0.8) and a random 10% re-exam. If the historical data cannot be re-collected, conduct a contemporaneous baseline now and follow forward - accepting a delay is better than publishing an artifact.
  2. Standardize participation. Either examine a random sample of enrolled kindergartners, or report results stratified by participation status, or weight for non-participation. Compute dmft on a fixed denominator.
  3. Measure exposure, not just residence. Water source (some children may attend childcare or live in parts of the county still on fluoridated systems), bottled-water use, fluoride dentifrice use, and any fluoride varnish or supplement receipt.
  4. Adjust for confounders at the child level: age, insurance, prior caries, country of origin, English proficiency, and receipt of varnish/sealants.
  5. Pre-specify the hypothesis and the lag. Fluoridation cessation effects on primary dentition should be visible within 3–5 years; state that window in advance so the result is not chosen after seeing the data.

Also useful: a natural-experiment analysis using the half of state cities that still fluoridate as a staggered comparison, plus a secondary analysis of Medicaid claims for restorative procedures as a corroborating outcome that is less sensitive to examiner drift.

3. Write the claim the board can defensibly make in its press release, and name the two sentences it must not print.

Read the worked answer

What the board can say "Between 2016 and 2021, the measured mean dmft among kindergarten children seen by our clinic rose from 1.8 to 2.6. During that same period, however, our screening method, examiners, participation rate, and access to care all changed, so this figure cannot by itself be attributed to the 2016 decision to stop water fluoridation. We are initiating a controlled comparison with neighboring cities that still fluoridate, using calibrated examiners and a standardized index, and will report results within X years. In the meantime, we recommend continued fluoride varnish for children at elevated risk and support for water fluoridation as a population-level measure with a strong evidence base."

Two sentences that must not print

  1. "Caries increased 44% because fluoridation was stopped." This asserts a causal attribution the design cannot support and converts an uncontrolled pre-post difference into a headline number. A percentage change on a moving denominator with changed examiners is not an effect size.
  2. "Our data prove the science of fluoridation." Do not stake the credibility of a well-established public health measure on a weak local dataset. If the number later proves to be an artifact, the retraction damages the measure itself - which is exactly the outcome a careful board should fear most.

A third, subtler trap: avoid saying "the data are inconclusive, so fluoridation doesn't matter." Absence of a valid local estimate is not evidence of no effect; the correct statement is that this particular comparison cannot answer the question, and a better one is underway.

Practical recommendation to pair with the release Because children are accumulating caries now while the study runs, recommend the interventions that do not depend on resolving the causal question: school-based fluoride varnish at least twice yearly for at-risk children, sealants on first permanent molars, a fluoride dentifrice message, and a referral pathway through the new Medicaid-serving clinic. Deferring all action until the study concludes converts methodological rigor into a delay that children pay for.