Item 7 and the minority key
At the item-review meeting for the March STEP Reading paper, the Grade 10 coordinator tables Item 7. The passage describes a city's rooftop beekeeping and says municipal rules 'lag behind' practitioners and are 'well-meaning'. The item asks what the writer implies about those rules. The key is D: 'Municipal rules are unnecessary.' In a sample of 180 candidates, 62% chose B: 'Municipal rules are well-intentioned but ineffective'; 24% chose D; the rest split between A and C. Among the top quartile by total reading score, B was chosen by 71%. One teacher argues the item is fine because the key was agreed at moderation; another wants it voided. The paper is due to be reused in June for a parallel class, and the school's placement decisions depend on the section score.
1. Using the distractor distribution and the textual evidence, decide whether Item 7 is defective. Show your reasoning.
Read the worked answer
Test for a single defensible key
- The text gives two signals: rules lag behind practice (a gap between rule and reality) and well-meaning in scare quotes (ironic distance). Both point to 'well-intentioned but ineffective' - that is option B.
- Option D requires the writer to say rules should not exist. The passage says they are out of step, not that they are pointless. D overstates.
Read the statistics as evidence, not decoration
- With four options, chance is 25%. The key attracts 24% - indistinguishable from guessing.
- The strongest readers choose B at 71%. When the top quartile systematically rejects the key, the usual explanation is a flawed key, not a weak cohort.
- A and C mop up the low scorers; that pattern is normal and tells you nothing either way.
Conclusion: the item is defective. Two options are defensible and the key is the weaker of the two. The fact that moderation agreed the key is irrelevant - moderation checked the key against the writer's intention, not against the full option set.
2. What scoring action do you take for the March cohort, and what do you tell the June class?
Read the worked answer
Scoring action
- Credit both B and D. Do not simply switch the key to B: candidates who chose D reasoned from the text ('lag behind' read as 'rules are behind the times, therefore dispensable') and should not be penalised for a distinction the item failed to draw.
- Recompute section scores with the dual key and re-issue placement decisions for any candidate whose band changes. Flag those cases explicitly to the placement panel.
Why not void outright Voiding removes 1/30th of the section from everyone equally but also removes the discrimination the item does provide between B/D choosers and A/C choosers. Dual credit preserves that.
What to tell June
- Do not reuse the item. Rewrite or replace it; a parallel class sitting the same flawed item creates a second cohort needing re-scoring.
- Tell students plainly that one item was found to have two defensible answers and that no one's score fell as a result. Ambiguity handled quietly breeds distrust; ambiguity handled openly does not.
- Log the item, the statistics and the decision in the item bank so the same stem is not rewritten the same way next year.
3. Rewrite the item so that exactly one option is defensible. Explain why each distractor now fails.
Read the worked answer
Rewrite
The writer places 'well-meaning' in quotation marks mainly to A. praise the officials who drafted the rules. B. suggest the rules are sincere but out of step with practice. C. show that the rules were written without any consultation. D. argue that the rules should be repealed.
Why only B survives
- A contradicts the ironic marking; a reader who takes the quotes at face value has misread the signal, and that is a legitimate comprehension error rather than a coin flip.
- C adds a cause (no consultation) the passage never mentions - it is checkable against the text.
- D needs 'repealed', which is a speech act the writer never performs; 'lag behind' does not entail repeal.
Design principle Each distractor now fails for a different, nameable textual reason: contradicted, unsupported, overstated. The original item failed because B and D differed only in degree of criticism, which the text does not calibrate - that is a writer's-intention problem, not a reader's.