Training Records Show Competence the Simulator Does Not
A licensed reactor operator has a complete training file: all required simulator hours signed off, annual requalification passed, and no failed evaluations in six years. During a routine simulator session observed by a new training instructor, the operator performs well on scripted evolutions but freezes when the instructor introduces an unscripted compound failure-loss of a feedwater pump combined with a stuck-open relief valve. The operator repeatedly attempts the same recovery action, does not verbalize, and does not call for help until prompted. Afterward the operator says, 'We never practice that combination.' The instructor reviews the file and finds that the last four annual evaluations used nearly identical scenarios, all passed at the minimum required score. The training department is measured on pass rates and schedule completion.
1. Evaluate whether this operator is competent, and critique the evidence the training system has been using.
Read the worked answer
The operator is competent for the tasks that have been trained and assessed-scripted, familiar evolutions-but there is no evidence of competence for unscripted compound failures, and the observed freeze is affirmative evidence of a gap. The training file measures exposure and schedule compliance, not capability. Four annual evaluations using nearly identical scenarios constitute a validity failure: the assessment has become predictable, so passing it demonstrates recognition of a known scenario rather than transferable skill. Passing at the minimum score every time is a warning sign, not reassurance; it suggests the bar is calibrated to the schedule rather than to required performance. The operator's own statement-'we never practice that combination'-is an accurate diagnosis of a curriculum gap. The training department's pass-rate and completion metrics create a perverse incentive: instructors are rewarded for throughput, so scenario diversity and honest failure are discouraged. The freeze itself (perseveration on a failed action, no verbalization, delayed call for help) is a recognizable pattern under novel high-load conditions, exactly what training should inoculate against.
2. Design a revised assessment and training approach that would produce credible evidence of competence.
Read the worked answer
Assessment: replace repeat scenarios with a validated scenario bank covering the full range of required competencies, including unscripted compound failures, and rotate so no operator sees the same scenario twice within a defined cycle. Assess against behavioral criteria-diagnosis quality, prioritization, verbalization, use of procedures, call for help, workload management-not just final plant state, and use at least two assessors for high-stakes evaluations. Include a deliberate 'unknown unknown' element to test transfer. Remove pass rate and completion from instructor performance metrics; measure instead scenario coverage, inter-rater reliability, and the proportion of evaluations that surface a genuine development need. Training: add a curriculum strand on compound and novel failures, teach and practice explicit diagnostic strategies (state the hypothesis, seek disconfirming evidence, time-box the attempt, escalate), and rehearse calling for help as a trained, rewarded behavior. Use post-session debriefs focused on reasoning. Finally, treat the operator's gap as a system finding: report it through the corrective action program, and protect the instructor from retaliation for surfacing it.