Training Evaluation and Program Improvement
Close the loop
Evaluation determines whether training closed the gap identified in the needs assessment, and it is designed alongside the objectives, not tacked on at the end.
Kirkpatrick's four levels are the backbone: Level 1 reaction (satisfaction, relevance, logistics), Level 2 learning (knowledge, skill, or attitude change, measured by pre/post tests, skills demonstrations, scored checklists), Level 3 behavior (on-the-job application, measured by observation, supervisor reports, audit data), and Level 4 results (incidents, injuries, citations, downtime, cost). Phillips adds Level 5, ROI, comparing monetary benefits to program cost.
Formative versus summative. Formative evaluation happens during design and delivery - pilot feedback, item review, mid-course adjustment. Summative evaluation judges the finished program.
Test quality matters. A criterion-referenced test measures against the objective's criterion, not against other learners. Items should be valid (measuring the stated objective) and reliable (producing consistent results). Item analysis flags questions everyone answers correctly (too easy, no discrimination) or that strong learners miss (possibly miskeyed or ambiguous).
Trap: a "smile sheet" (Level 1) offered as evidence that learning occurred. Another trap: assigning Level 4 results to a program too small or too remote to influence them.
Rule of thumb: match the level to the objective - reaction for satisfaction and logistics, learning for knowledge and skill objectives, behavior for transfer, results for organizational impact.
Worked example: A LOTO course earns strong reaction scores yet repeat violations appear six months later. Action: move to Level 3, observe lockout performance in the field, and check whether supervision enforces the procedure - the residual gap may be consequence-based rather than instructional.