When the Answer Key Is the Problem
You coach 30 Class 10 students for the reasoning section of a scholarship test. In yesterday's 25-item mock, item 14 was a six-person seating puzzle: six friends A–F sit in a row facing north; A sits third to the left of D; B sits immediately right of A; C is not at either end; the question asks who sits second to the left of F. The printed key says (b) E; 11 students chose (d) C. On re-solving, the clues fix A, B, D and C but leave E and F interchangeable in the last two seats, so the question has no unique answer and both (b) and (d) can be defended. Item 22 asked: in a certain code TRAIN is written as USBJO; how is RIVER written? The key says SJVFS; the correct string is SJWFS. Twenty-seven of 30 students got item 22 'wrong'. Students now say reasoning questions are guesswork, and two have stopped attempting puzzles.
1. Decide the marking action for both items and justify it to students and to a principal who wants the results today.
Read the worked answer
Item 14 - double-key. Because A, B, D and C are fixed and E and F can swap, both (b) and (d) follow from the clues. Award full credit to every student who chose either. The item tests guessing, not reasoning.
Item 22 - re-key to SJWFS. The code is a simple +1 shift: T→U, R→S, A→B, I→J, N→O, so RIVER → S J W F S. The printed key SJVFS swaps the second and third letters. All 27 who wrote SJWFS get the mark; also check whether anyone wrote the key's SJVFS and deserves credit for reproducing its slip.
Recompute the totals out of 25 and reissue the ranks. If item 14 is dropped instead of double-keyed, the denominator changes to 24 and every rank shifts - that is a defensible alternative, but choose one and state it.
To the students: say plainly that the paper was wrong, not that the class was weak, and show the two valid arrangements on the board. To the principal: give the corrected distribution and the cause in one line - no independent verification of the key. A wrong rank published on time is worse than a correct one published an hour later.
2. Teach students how to handle a possibly under-determined puzzle in the real exam, within 90 seconds, without losing confidence in the section.
Read the worked answer
- Draw the row as six boxes and mark each position as fixed or free. If the person asked about sits in a free box, the item cannot have a unique arrangement as set - but check for a missed constraint first: facing direction, 'immediate' versus 'somewhere', 'only', 'not adjacent', or a clue read too quickly.
- Re-read the options. If exactly one option survives every valid arrangement, that is the answer even when the arrangement is not unique - the question may be about an invariant ('who is at an end?'), not the full seating.
- If two options both survive, the item is flawed. In the real exam, choose one and move on within 90 seconds; proving the paper wrong earns no marks.
- Rebuild confidence with practice: give 10 items in which one clue is deliberately missing, and ask students to state which clue is needed and why. This converts 'reasoning is guesswork' into 'this question is under-determined, and I can prove it' - a reasoning skill in itself.
- Model the language: 'The clues do not fix F's seat, so I will take the option that is forced in both arrangements.'
3. Repair item 14 with a minimal change so it has a unique answer, and set out the process failure behind item 22 and how you will prevent it.
Read the worked answer
Minimal repair of item 14: add one constraint that removes the E–F swap - 'F sits at the right end' (or, equivalently, 'E sits immediately left of F'). Every other clue and the option set stay the same, so the item now has one arrangement and one answer and can be reused in the next mock.
Why item 22 failed: the key was set once, by one person, in a hurry; a single transposition passed as correct because nobody checked the string letter by letter.
Process to adopt before the next paper
- Two independent solvers per item, neither seeing the other's answer.
- For coding items, decode in a column (R→S, I→J, V→W, E→F, R→S) and compare the full string, not its look.
- Anticipate the common slip in the distractors: here, swapping two letters is exactly the error a student would make, so an option built on it will attract choosers - and can expose a wrong key.
- Pilot the paper on one strong student or a colleague under time; a 25-item paper set and keyed by one teacher is a single point of failure.