The Item That 41 Percent Answered Better Than the Key
Your seventh-grade Texas history team - four teachers, 132 students - reviews the common assessment. One item reads: "Which of the following was the most important cause of the Texas Revolution?" with four options. Thirty-four percent chose the key; 41 percent chose the distractor naming economic tensions tied to slavery. Two teachers want to reteach the unit; one wants to drop the item; the district requires the item to be administered as written. The coach asks you to lead the decision. You have the data report, the answer documents, and 25 minutes at Wednesday's PLC.
1. Identify three competing explanations for the response pattern and specify what evidence would let the team distinguish among them within the 25 minutes.
Read the worked answer
The pattern is consistent with at least three different problems, and they call for different responses:
- Item flaw. "Most important cause" asks students to rank causes by weight, which is a historiographical judgment, not a fact. The distractor is a recognized long-term cause; a student who knows more may reasonably prefer it. Evidence: ask the team to defend the key without appealing to the textbook's ordering, and check whether a knowledgeable adult could choose the distractor and be wrong only by the test's fiat.
- Instructional emphasis. If the unit emphasized immediate grievances (centralized power, 1835–36 actions) and treated economic factors briefly, the distractor would be genuinely unfamiliar. Evidence: scan the unit's lessons and notes for coverage of the distractor's content; check whether the class that covered it most scored differently.
- Item language or reading load. "Most important" plus four long options may be measuring reading rather than history. Evidence: look at whether emergent bilingual students are over-represented among distractor choosers, and read the options aloud to a small group and re-poll.
With 25 minutes, do (1) by argument, then pull eight answer documents stratified by class and check the demographic pattern for (3). Student interviews come after the meeting.
2. Recommend a course of action for the meeting and justify it against the two proposals on the table.
Read the worked answer
Recommendation: administer the item as required, but treat it as a discussion text rather than a valid measure, and flag it for revision in the bank.
Why not reteach the whole unit: reteaching 132 students on the strength of one item, before you know whether the item or the instruction is at fault, spends days you cannot recover and may reinforce the wrong content.
Why not drop it: the district requires it, and dropping it hides the problem from the people who maintain the bank.
What to do instead:
- Administer it, then spend 15 minutes having students argue for two different causes using specific evidence. The discussion itself demonstrates that more than one answer is defensible - which is the actual finding.
- Rewrite the item for the next cycle so it tests a defensible target: "Which of these was an immediate cause of the Texas Revolution?" or "Which cause is identified in the excerpt?" Both have keys that do not depend on ranking.
- Reteach only the specific content gap, targeted to the students whose interviews or notes show they never encountered the economic factors - not the whole unit.
- Document the flag in the item bank so the next team does not rediscover it.
Rationale to the team: assessment data is evidence about a system of item plus instruction plus student; when an item is not defensible, its results cannot indict the teaching.