When 31 of 38 Online Quiz Scores Are Identical
Northside Fire Rescue requires 38 probationary firefighters to complete a 20-question LMS quiz drawn from a 25-question bank. After the window closes, 31 members score 95–100%, while 7 score 60–80%. Two members who scored 100% selected the same incorrect answer on question 14 and submitted within 90 seconds of each other; both logged in from the same station Wi-Fi, which is common. The LMS allows unlimited attempts and does not randomize questions. Six practical sign-off sheets for hose advances lack an evaluator signature, though names are typed. The department’s training records policy requires scores, date, instructor, and remediation within 30 days. A union representative asks whether the quiz is valid. The training chief wants to “just curve it” and move on.
1. Is the quiz result valid enough to certify the probationary firefighters? Analyze the evidence and decide what to do with the scores.
Read the worked answer
The scores are not valid enough to certify on their own. Validity here means the scores support the inference that each member knows the material. Several features undermine that inference: unlimited attempts without item randomization; a small bank so members can share questions; a cluster of near-perfect scores; and two identical wrong answers submitted within 90 seconds from a shared station network. Shared Wi-Fi alone is not proof of misconduct, but the pattern is a test-security red flag that must be examined.
The practical sign-off sheets are a separate validity problem. Typed names without evaluator signatures do not prove that a qualified evaluator observed the hose advance. If the department treats those sheets as certification evidence, the record is incomplete regardless of quiz scores.
I would not curve the scores; a curve would conceal the problem. I would suspend certification based on this quiz, preserve the LMS logs, interview the two members and the evaluators, and re-administer a proctored, randomized assessment. The seven low scores also need remediation, but the high scores cannot be accepted until security is restored. The union representative should receive the assessment rationale, not the members’ personal data.
2. What corrective actions would you take for test security, records, and remediation, and in what sequence?
Read the worked answer
Sequence matters because remediation before a valid assessment wastes time and a re-test before investigation can destroy evidence.
1. Preserve and investigate. Export LMS logs, attempt histories, item responses, and timestamps. Interview the two members separately and the six practical evaluators. Do not accuse; ask how the quiz was taken and who observed the hose advance.
2. Fix the assessment. Randomize item order and answer order, limit attempts, require proctoring or a lockdown browser, and use a larger bank with scenario items. Set a defensible passing standard before re-administration. If the quiz is low-stakes knowledge check, say so; if it is certification, treat it as such. 3. Fix the practical records. Re-run the hose advance with a qualified evaluator present and a signature on the sheet. Add an evaluator ID and date field. Anyone who cannot demonstrate the skill gets remediation. 4. Remediate and re-assess. The seven low scorers receive targeted remediation; the 31 high scorers also re-take the secured quiz because their scores are not trustworthy. 5. Correct the records. Mark the original scores as “invalidated-test security” and file the new results with dates, instructor, and remediation notes. Do not erase the originals; retain the audit trail.
3. How would you communicate the decision to the training chief and the union representative, and what would you put in writing?
Read the worked answer
To the training chief: state the risk plainly. “Curving” the scores would create a record that says 38 members were certified by an assessment that allowed unlimited attempts, shared items, and no proctoring. If one of those members later fails on the job, the department cannot defend the training record. The recommendation is to invalidate the quiz results, re-administer a secured assessment, and correct the six unsigned practical sheets. Include the timeline and the operational impact-probationary members may need extra shifts or a delayed sign-off.
To the union representative: explain that this is an assessment-integrity action, not discipline. The shared Wi-Fi is common and is not being treated as proof of cheating; the issue is that the assessment cannot distinguish individual knowledge. Provide the process: notice, proctored re-test, remediation, and an appeal route. Keep individual members’ responses confidential.
In writing, record the decision, the evidence reviewed, the standard applied, the corrective actions, the re-test date, and who approved the change. For the six practical sheets, record that the original signatures were missing and that re-evaluation occurred. The written record should show that the department protected both certification integrity and due process.