The Ambiguous Key: Deciding an Item's Fate With No Time to Re-Trial
You are the language reviewer for a practice question bank used by candidates preparing for the TS EdCET General English section. A fourteen-line passage describes a retired postmaster who saves money for two years before reopening a closed village library, and who then lends books without deposit to anyone who asks. Item 7 reads: 'The postmaster's decision to reopen the library can best be described as - (A) impulsive (B) calculated (C) reluctant (D) charitable.' The key is (B). In a trial with 240 candidates the item's difficulty was 0.28 and its point-biserial correlation with total score was 0.06; 61 per cent of the top third chose (D). The item survived a first review because the word 'calculated' appears in the passage's second paragraph and because 'charitable' had been dismissed as too easy a word. The editor now asks you to settle the item's fate before the file goes to print in three days, and there is no budget for a second trial.
1. Read the three statistics together and state precisely what is wrong with Item 7. Explain why the appearance of the word 'calculated' in the passage does not defend the key.
Read the worked answer
The three figures must be read together, because each alone is inconclusive:
- p = 0.28 - only about one candidate in four answered correctly. For a four-option item, random marking alone yields roughly 0.25, so the item is performing barely above chance.
- Point-biserial = 0.06 - the item does not separate strong readers from weak ones. A sound inference item normally correlates positively with total score; a near-zero value means success on this item is unrelated to reading ability.
- 61 per cent of the top third chose (D) - when the ablest candidates converge on a distractor, the likely explanations are that the key is arguable, or that the passage does not warrant the key strongly enough. This is a distractor-analysis finding, not a difficulty finding.
'Calculated' appearing in paragraph two is a lexical match, not evidence of comprehension. An item whose defence is that the key word is visible in the text is testing scanning. Moreover, the no-deposit lending detail genuinely supports 'charitable', so the item has two defensible answers - a keying flaw that exists independently of the statistics.
2. Decide whether to keep, revise or drop the item, and justify your decision against the real constraints (three days, twelve items in the section, no re-trial).
Read the worked answer
Decision: revise, with a drop as fallback if the revision cannot be secured.
Reasoning against each option:
- Keep as it stands is indefensible. A near-zero point-biserial plus a contested key means the item penalises the candidates it should reward, and a practice bank's only product is trust in its keys.
- Drop outright is safe but wasteful here, because the passage contains enough explicit detail to support a cleanly keyed item. Dropping should be reserved for the case where no option can be made entailed by the text.
- Revise is possible within three days because revision needs no new pilot if the revised item satisfies a stricter standard than the original.
That standard should be: the correct option can be justified by pointing to one stated sentence, and every distractor is contradicted by another stated sentence. This replaces pilot evidence with textual entailment - the only defensible substitute when no re-trial is possible. Avoid the temptation to rewrite using another inference: an untested inference item risks repeating the same ambiguity.
A secondary safeguard: if the revised item still cannot be falsified distractor-by-distractor at the desk, drop it and substitute a vocabulary-in-context item on the same passage, which is far easier to key unambiguously without a trial.
3. Write the revised stem and options, and state the textual evidence that makes each distractor wrong.
Read the worked answer
Revised item
The postmaster's two years of saving before reopening the library mainly show that he - (A) acted on a long-formed plan (B) hoped to recover his money from borrowers (C) was pressed into it by the villagers (D) had lost interest in his old occupation
Key: (A)
Falsification of each distractor against the text:
- (B) is contradicted by the statement that he lends without deposit and charges nothing; no recovery mechanism exists in the passage.
- (C) is contradicted by the sequence, in which the villagers learn of the library only after he reopens it; the initiative is his.
- (D) is contradicted by the retirement detail and by the fact that he spends his own savings on the library - an act of continuing interest, not withdrawal.
Why this fixes the original fault. The key now rests on a single, stated fact (two years of deliberate saving) rather than on an adjective. Option (A) no longer competes with a second defensible reading, because 'charitable' has been removed as an option and cannot be argued for. Each distractor is textually falsifiable rather than merely 'less good'. Finally, the stem asks for a supported inference, so the item cannot be answered by locating a word.