Training Records, Evaluation, and Testing
Domain focus
This domain covers documentation, evaluation of instruction, and test quality. Instructor II is expected to evaluate instructors and programs, and to keep records that are accurate, secure, and defensible.
Core rules
- Records include attendance, skills checklists, test scores, certifications, remediation, and instructor qualifications. Retain them per AHJ, state, or provincial policy; protect confidentiality.
- Formative evaluation occurs during instruction; summative evaluation occurs at the end. Use both.
- Kirkpatrick levels: reaction, learning, behavior, results. Reaction alone does not prove competence.
- Test validity means the test measures the objective; reliability means consistent results.
- Item difficulty p = number correct ÷ number tested. Item discrimination compares high-scoring and low-scoring groups.
Worked example
Twenty students take a 25-item quiz. Item 7 is answered correctly by 16 students: p = 16 ÷ 20 = 0.80. If the top five students mostly answer it correctly and the bottom five mostly miss it, discrimination is strong. Reasoning: p = 0.80 is moderately easy but acceptable for a safety-critical fact; a negative discrimination index suggests a confusing or miskeyed item and requires review.
Traps
Recording only final scores; leaving skills sheets unsigned; using an unreliable checklist; evaluating only reaction; treating a high pass rate as proof of good instruction; failing to document remediation.