Training Needs Analysis and Program Evaluation
Domain focus
Training needs analysis and program evaluation tests whether the Instructor III can prove that training is needed and whether it worked. Expect items on organizational, task, and individual needs analysis; gap analysis; data sources; formative and summative evaluation; and Kirkpatrick levels.
Core rules
- Separate a want from a need: define the current performance, the required performance, and the gap.
- Use multiple data sources: incident reports, after-action reviews, observation, audits, certification records, surveys, and stakeholder interviews.
- Analyze at three levels: organization, task, and individual. A task gap may be a policy, equipment, or staffing problem, not a training problem.
- Evaluate at reaction, learning, behavior, and results; use the level that matches the decision.
- Feed findings into revision, budget, and instructor development; document the method and limitations.
Worked example
Low-air emergency reports rise, so a captain requests a new SCBA class. The Instructor III first checks the baseline: reports per 100 calls, by shift and crew. Observation and after-action reviews show delayed recognition, not missing knowledge. The task analysis identifies a decision and radio-report gap. A short scenario course is built with pre- and post-scenario scoring; behavior is tracked for 90 days. Reaction sheets are used only for logistics. Reasoning: the evaluation design follows the suspected gap and the decision the chief must make.
Traps
Relying on end-of-class smile sheets as proof of results; skipping baseline data; treating every problem as a training problem; using a convenient sample without saying so; and reporting scores without linking them to the original gap.