Evaluating Learning Impact
Evaluation is planned, not bolted on
Know the four levels of the Kirkpatrick model: reaction, learning, behavior, and results. Understand that each level answers a different question and that higher levels require more rigor. Recognize the New World Kirkpatrick additions emphasizing required drivers and learner confidence/commitment. Be able to contrast Kirkpatrick with the Phillips ROI model, which adds a fifth level (return on investment) and emphasizes isolating the effects of training.
Formative and summative
Formative evaluation improves the program during design and delivery; summative evaluation judges outcomes after. Know the difference between validity (does the assessment measure the objective?) and reliability (does it produce consistent results?).
Data and attribution
Candidates should understand pre/post designs, control groups, and the challenge of isolating training's contribution from other factors. Recognize techniques like trend line analysis, control groups, and participant estimation with adjustment. Know that Level 3 (behavior) usually requires follow-up measurement on the job and often input from supervisors.
Traps
Traps include treating a smile sheet as proof of learning, claiming business results without isolating causes, and confusing an evaluation plan with a survey. Another trap is assuming ROI is always required-sometimes it is not feasible or proportionate.
Rule of thumb
Match the evaluation level to the stakes: high-cost, high-visibility programs justify Level 3 and 4 measurement; low-stakes awareness sessions may stop at Level 1 and 2. Worked example: a sales negotiation program shows strong reaction and learning scores. To claim behavior change, collect manager observations 60 days post-training; to claim results, compare sales-cycle metrics against a matched comparison group rather than attributing all improvement to training.