Scientific Foundations, Research Methods, and Outcome Evaluation
What to master
Scientific literacy is required for selecting and evaluating treatments. Know research designs: RCTs, quasi-experimental designs, single-case experimental designs, longitudinal and cross-sectional studies, and program evaluation (formative versus summative). Understand internal, external, construct, and statistical conclusion validity; threats such as maturation, history, regression to the mean, attrition, and selection bias. Distinguish mediation (how an effect occurs) from moderation (for whom or under what conditions). Know effect sizes (Cohen's d, eta squared), confidence intervals, number needed to treat, statistical versus clinical significance, reliability versus validity, and measurement invariance. In outcome evaluation, use measurement-based care, intention-to-treat analysis, and quality improvement methods.
Traps
Common errors: interpreting correlation as causation; confusing mediation and moderation; ignoring regression to the mean when a distressed group improves; overinterpreting subgroup analyses; and treating statistical significance as clinical importance. Another trap is assuming a reliable measure is valid.
Worked example / rule of thumb
A school reports better behavior after a new program, but there was no control group and the program began after a difficult semester. Maturation, regression to the mean, and history threaten the conclusion. Rule of thumb: ask 'compared with what?'-without a comparison condition, improvement is not proof of cause.