Deciding whether to buy a laser: appraising a manufacturer-funded trial
The principal of your practice wants to buy a diode laser costing about 4,000 KWD because a sales representative has presented a study. The study recruited 24 patients in a split-mouth design: each patient received scaling and root planing plus laser in one quadrant and scaling and root planing alone in the contralateral quadrant, with 6 months of follow-up at a single centre. The trial was funded by the laser manufacturer. It reports a mean additional probing depth reduction of 0.3 mm in the laser group (p = 0.04) and no difference in clinical attachment gain; bleeding reduced in both groups. The proportion of sites achieving at least 2 mm probing depth reduction was 34% with laser and 26% with scaling alone. No confidence intervals are reported, adverse events are not described, and smokers were excluded. The principal asks: 'So should we buy it?'
1. Appraise the study's internal validity and interpret the p-value and the effect size. What does the 34% versus 26% difference mean in practical terms?
Read the worked answer
Internal validity concerns:
- Split-mouth design: quadrants in the same mouth share plaque, saliva, and host response. Contamination between sides is possible, and sites within a patient are not independent. Analysis must therefore be multilevel or patient-level; if the paper analysed sites as if independent, the p-value is artificially small.
- Single centre, short follow-up (6 months), small sample (n=24) - underpowered for safety outcomes and for detecting rare harms.
- Industry funding does not invalidate the study, but it is a risk factor for selective reporting; absence of confidence intervals and of adverse events is a reporting deficiency.
- Smokers excluded means the result may not transfer to your patient population.
- No patient-reported outcomes, no cost data.
Interpreting the statistics: p = 0.04 means that if there were truly no difference, a result this extreme would occur about 4 times in 100. It says nothing about size or importance. A 0.3 mm additional probing depth reduction is below any reasonable threshold for clinical importance and is within measurement error for a periodontal probe. The absence of a confidence interval is serious: without it we cannot judge the precision of the estimate or whether the lower bound is close to zero.
The 34% versus 26% figures give an absolute risk reduction of 8 percentage points. Number needed to treat = 1 ÷ 0.08 = 12.5, i.e. about 13 sites must be lasered for one extra site to gain 2 mm. Combined with 4,000 KWD of capital cost, consumables, and training time, that is a weak basis for adoption.
2. The representative then cites a subgroup of 8 smokers in the same trial showing 1.2 mm additional probing depth reduction (p = 0.03), and a meta-analysis reporting a pooled weighted mean difference of 0.28 mm (95% CI 0.05 to 0.51) with I² = 78%. How do you interpret these?
Read the worked answer
The smoker subgroup: subgroup analyses in small trials are fragile. With 8 patients, the estimate is imprecise, and the result is plausible by chance - particularly when multiple subgroups were likely examined and only the significant one reported. It also contradicts the wider literature, in which smoking usually worsens periodontal outcomes, not improves them. The subgroup is hypothesis-generating at best, and should not drive a purchase decision. It should also be checked whether the trial was powered for this comparison at all.
The meta-analysis: the pooled difference of 0.28 mm is statistically significant because the 95% CI (0.05 to 0.51) excludes zero. But statistical significance is not clinical significance: the entire confidence interval lies well below a clinically meaningful threshold, and the upper bound of 0.51 mm does not represent a benefit a patient would notice. I² = 78% indicates substantial heterogeneity, meaning the trials disagree with each other - pooled estimates should then be interpreted cautiously, and sources of heterogeneity (laser type, wavelength, follow-up, adjunct use, population) explored. If the pooled estimate is dominated by small, industry-funded studies, publication bias is also likely.
The correct interpretation is that the adjunct may produce a very small average improvement that is unlikely to change outcomes, at a cost - which is a decision about value, not just efficacy.
3. Draft the recommendation you would give the principal, and describe how you would evaluate the technology if the practice decided to adopt it anyway.
Read the worked answer
Recommendation: do not purchase on the basis of this evidence.
- The best available estimate of benefit is a fraction of a millimetre of additional probing depth reduction, with no demonstrated attachment gain and no evidence of patient-relevant benefit such as tooth survival, comfort, or reduced treatment time.
- The evidence base is short-term, small, heterogeneous, and largely industry-funded, with no safety reporting.
- The capital and running costs are substantial and could fund interventions with better evidence - smoking cessation support, hygiene time, better maintenance recall, or staff training in non-surgical debridement.
- The study does not address the clinical question that matters: does the adjunct change outcomes in a way the patient would value compared with high-quality mechanical debridement alone?
If the practice adopts it anyway, treat it as a local evaluation, not an evidence claim:
- Define the clinical question and outcomes in advance (probing depth change, bleeding scores, patient-reported discomfort, time, cost per site).
- Register or at least document the protocol; record all treated patients, not just successes.
- Use a consistent baseline and re-evaluation interval (e.g. 3 months) and a calibrated single examiner where possible.
- Audit adverse events and complaints.
- Set a predefined decision rule: if the audit does not show a clinically meaningful benefit, discontinue.
The professional and ethical point is that adoption should follow evidence and honest communication with patients - a laser must not be marketed as a superior treatment on the strength of a 0.3 mm average difference.