I85PREP
⌘ K
EBP-EAP study materials
EBP-EAP study resources

EBP-EAP: Critical Appraisal of Research Evidence

EBP-EAP · Work through your exam, one topic at a time.

All guide sections

Critical Appraisal of Research Evidence

What this domain tests

Critical appraisal items ask you to judge whether a study’s results are valid, what the results are, and whether they apply to your patients. You need to recognize bias, interpret effect sizes and confidence intervals, and choose an appraisal tool.

Core rules

  • Appraise three questions: Is the study valid? What are the results? Will the results help locally?
  • Internal validity concerns bias and confounding; external validity concerns generalizability. Reliability concerns consistency of measurement.
  • Common biases: selection, performance, detection, attrition, reporting, and publication bias. Randomization, allocation concealment, blinding, intention-to-treat analysis, and complete follow-up reduce bias.
  • Effect measures include RR, OR, and HR for binary outcomes; mean difference or SMD for continuous outcomes; NNT/NNH for clinical impact. A confidence interval gives a range of plausible effects; a p-value tests compatibility with the null, not the size or importance of an effect.
  • Statistical significance is not clinical significance. A large trial can produce a tiny, unimportant difference with p < 0.05.
  • Use design-specific tools: RoB 2 for randomized trials, ROBINS-I for nonrandomized studies, CASP or JBI checklists, AGREE II for guidelines, and AMSTAR 2 for systematic reviews.
  • For meta-analysis, inspect heterogeneity (I², τ²), forest plots, and funnel plots for publication bias.

Worked example

An RCT reports that a new dressing reduced surgical site infection from 12% to 9% (RR 0.75, 95% CI 0.60–0.94, p = 0.01). Appraisal: allocation concealment and blinding were adequate; 8% loss to follow-up; intention-to-treat analysis used. The result is statistically significant and the CI excludes no effect. Absolute risk reduction is 3 percentage points, NNT ≈ 33. Before adopting, check whether your unit’s baseline rate and patient mix are similar; a 3-point reduction may be worthwhile but not if cost or harm is high.

Traps that cost marks

  • Calling a study valid because p < 0.05.
  • Ignoring confidence intervals and absolute effects.
  • Confusing odds ratios with risk ratios.
  • Accepting subgroup findings without interaction tests.
  • Forgetting that a systematic review can itself be biased.