Evaluation
What you must know
Evaluation judges merit and worth. Learn the types cold: formative (improve the program while it runs: needs assessment, pilot testing, process monitoring) and summative (judge at the end: impact and outcome). Know process (was it delivered as intended - reach, dose, fidelity), impact (immediate change in behavior, attitudes, or knowledge), and outcome (longer-term health status and quality-of-life change).
Know the CDC Framework's six steps: engage stakeholders, describe the program (logic model), focus the design, gather credible evidence, justify conclusions, and ensure use while sharing lessons. Evaluation standards stress utilization - an unused evaluation has failed regardless of rigor.
Know designs and validity threats: pre-experimental (one-group posttest), quasi-experimental (comparison group, no randomization), experimental (random assignment). Threats include history, maturation, testing, instrumentation, selection, attrition, and regression to the mean. Match the threat to the design flaw.
Know cost analysis: cost-benefit (dollars in, dollars out), cost-effectiveness (cost per unit of outcome, such as cost per quit), and cost-utility (cost per quality-adjusted life year).
Traps: calling a behavior change an outcome when it is an impact; confusing a process objective with a summative finding; failing to plan evaluation at the front end; reporting a statistically significant but practically trivial change as success.
Rule of thumb: Impact = behavior; outcome = health status. If the measure is diabetes incidence, it is an outcome.
Worked example: A worksite program reports 90% of sessions delivered and 85% attendance (process), 40% more employees meeting activity guidelines at six months (impact), and a 3-point drop in mean BMI at two years (outcome).