Research, Measurement, and Evidence-Based Practice
Evidence and study design
Know levels of evidence, the difference between efficacy (controlled conditions) and effectiveness (real-world), and basic designs: RCTs, quasi-experimental, pre-post, and qualitative approaches. Understand internal vs. external validity, confounding, selection bias, and the limits of pre-post designs without a comparison group. Traps include treating correlation as causation and assuming statistical significance equals practical significance.
Measurement
Understand reliability (consistency) versus validity (measuring what is intended), including content, construct, criterion, and face validity. Know scale types (nominal, ordinal, interval, ratio) and appropriate statistics. Recognize ceiling/floor effects, response bias, and the danger of using unvalidated instruments for high-stakes decisions.
Evidence-based practice
EBP integrates best available evidence, practitioner expertise, and client values and context-not evidence alone. In organizations, evaluate interventions with appropriate metrics before and after, and consider base rates and regression to the mean when interpreting improvement.
Rule of thumb
Ask three questions of any measure: Is it reliable? Is it valid for this use? Is the comparison fair? If any answer is no, treat findings as provisional.
Worked example
Engagement scores rise after a program. Before crediting the intervention, check for a comparison group, baseline trends, and regression to the mean-especially if the group was selected for low scores.