Item 27: A Reverse Distractor and a Grade Challenge
After a 60-item medical-surgical exam in a 64-student course, you review the psychometrics. Item 27 has a difficulty of 0.38 and a point-biserial of 0.11. Distractor analysis shows option C was selected by 41 percent of students in the top third and 9 percent of students in the bottom third. Option C reflects a common misconception about fluid shifting in heart failure. The keyed answer is B. Two students have challenged the item, arguing that C is also correct under a specific clinical condition. The blueprint requires four fluid and electrolyte items, and this is one of them. Course policy allows up to 5 points of item credit adjustment and requires faculty committee approval for any key change. The exam average is 71, and the program benchmark is 75. The next exam is in 12 days.
1. Interpret the item statistics, including the distractor pattern, and decide whether to keep, revise, or discard item 27. Justify your decision against the blueprint and course policy.
Read the worked answer
Reading the statistics. Difficulty 0.38 is hard but not automatically flawed. The more serious problem is the point-biserial of 0.11, which is below the usual 0.20 threshold and signals that the item is not discriminating between stronger and weaker students. The distractor pattern is the decisive evidence: option C attracted 41 percent of the top third and only 9 percent of the bottom third. That is a reverse or non-functioning distractor. High performers chose C for a reason, likely a defensible clinical nuance. This is not a simple bad option; it may be a second correct answer or an ambiguous stem.
Decision. Do not discard automatically, because the blueprint requires four fluid and electrolyte items and this is one of them. Do not keep it as scored in its current form. The defensible path is to pull the item from scoring pending review, revise the stem to remove the ambiguity (for example, specify the clinical condition and what is being asked), and replace it with a comparable blueprint item from the item bank for the current scoring cycle. Because changing the key requires committee approval and the policy caps adjustments at 5 points, I would not unilaterally key C. I would bring the item, the distractor analysis, and the students' rationale to the faculty committee with a recommendation: either double-key B and C if the stem genuinely supports both, or void the item and award credit to all students. Awarding credit to all preserves blueprint coverage in the bank but removes the item from discrimination; the committee should decide based on whether C is defensible.
Justification. Keeping a non-discriminating item with a reverse distractor penalizes strong students and undermines validity. Discarding without replacement weakens blueprint coverage. Revising after the exam does not help the current cohort unless credit is awarded. Therefore: remove from current scoring, seek committee approval for credit or a double key, and bank a revised version for future use.
2. Recommend a defensible response to the two student challenges and to the 71 average. Include what you would communicate, what you would not do, and how you would document the decision.
Read the worked answer
To the students. Acknowledge the challenge in writing within the policy timeline, explain that the item is under faculty review, and state the outcome after the committee meets. If C is judged defensible, tell them credit was awarded to all students, not just the two who challenged, to avoid inequity. Do not debate the clinical content in a group email or in class.
To the cohort about the 71 average. Do not curve the exam to reach the 75 benchmark. A curve hides the validity problem and breaks the link between grades and competence. Instead, share a brief, non-defensive summary: the item analysis identified one flawed item, credit was applied according to policy, and the remaining 59 items stand. Offer a structured review session focused on the blueprint areas with the lowest class performance. If the average remains below benchmark after the item decision, the course-level response is a formative quiz and targeted review, not a benchmark change mid-course.
Documentation. Keep the item statistics, the distractor analysis, the students' written challenge, the committee minutes, the decision rule applied, and the communication sent. Note the date and the policy section. This protects the program in a grade appeal and creates an audit trail for accreditation.
What not to do. Do not ignore the challenge, do not change the key alone, do not reveal other students' scores, and do not tell the class that the item was bad without a corrective action.