Evaluation and Outcomes
Judging whether it worked
Evaluation answers two questions: did the outcome change? and was the change attributable to what we did? Candidates must distinguish outcome measures (end results) from process measures (whether the plan was delivered as intended). Both are needed; outcome data alone cannot explain a failure.
Design and attribution
Know the basic logic of pre–post comparison, control or comparison groups, baseline adjustment, and the threats to inference: maturation, history, regression to the mean, selection, and measurement drift. A common trap is treating an observed improvement as proof of effectiveness when a plausible rival explanation exists. Another is confusing statistical significance with practical importance - a small effect can be significant with a large sample.
Reporting and learning
Evaluation should close the loop: findings feed back into planning. Report limitations honestly; selective reporting is an integrity issue as well as a methodological one.
Rule of thumb
Use the rival-explanation test: before crediting the intervention, ask what else could have produced this result. If a rival explanation is plausible and unaddressed, temper the conclusion.
Worked example
Scores rise after a programme, but participants were also selected for high motivation. The defensible conclusion is that improvement occurred, but attribution is uncertain because selection is an unaddressed rival explanation.