Reviewing Three Comprehension Items on a Mobile Library Passage
A B.Ed trainee on language-methods internship has drafted comprehension items on this passage for a Class 8 test:
'The district's mobile library van reaches Karanjgaon every second Saturday. In 2019 it carried 1,200 books; in 2023, 640. The librarian blames the van's ageing engine rather than any loss of interest, pointing out that footfall at the two static libraries in neighbouring villages rose by a third over the same four years. Yet the van's Saturday reading circle still draws about twenty children at every visit, almost the same number as in 2019.'
Her three items are: Q1. How many books did the van carry in 2019? Q2. The librarian's explanation rests on which assumption? Q3. Is the mobile library a success or a failure? Support your answer.
Her mentor must approve the paper by the evening. The trainee argues that Q1 should be kept because it is 'objective and easy to score', and that Q3 should be kept because it is 'higher-order thinking, which the new policy wants'.
1. Classify each of the three items as sound or defective for an objective comprehension test, and state precisely what each item actually measures.
Read the worked answer
Q1 - defective as a comprehension item. The answer (1,200) is stated verbatim in the second sentence. The candidate needs only to scan for a numeral; no linking of ideas is required. It is perfectly reliable and easy to mark, but it measures location of a stated fact, not comprehension of the passage's argument. It is acceptable only if the blueprint deliberately allots a small weight to literal detail.
Q2 - sound. 'Rests on which assumption' asks the candidate to reconstruct the librarian's reasoning: lower van use is caused by the engine, not by lost interest. To answer, the candidate must connect the falling book figures with the rising footfall in static libraries and the steady reading-circle attendance. A single best option is defensible.
Q3 - defective. It is globally evaluative, has no keyed single answer, and both verdicts can be defended from the text (failure: stock halved; success: attendance unchanged). For an objective test this creates marking disputes. If the test has a subjective section with a rubric - evidence of both sides, reference to the text - the item can be moved there and rewritten with that rubric.
2. Rewrite Q1 as an inference item and rewrite Q3 as a well-formed objective item, keeping the same passage.
Read the worked answer
Rewriting Q1 (inference, single key). 'The librarian's argument would be most weakened if it were found that:' (a) the van's engine has been repaired twice since 2019 (b) children in neighbouring villages now borrow books from a school collection instead (c) the reading circle meets only once a month (d) the van carried 1,300 books in 2018
Correct key: (b) - it supplies an alternative cause for the fall in van use, breaking the assumed link between the engine and the drop. Note that (a) is a distractor because repairs alone do not show the engine is not the cause, and (d) is irrelevant to the 2019-2023 comparison.
Rewriting Q3 (evaluative, keyed). 'Which statement is best supported by the passage?' (a) The mobile library has failed and should be withdrawn. (b) Falling book stock alone does not establish falling interest in reading. (c) The static libraries are better managed than the van service. (d) The reading circle has grown since 2019.
Correct key: (b). It is the only claim the passage's evidence supports; (a) and (c) overreach, and (d) is contradicted by 'almost the same number'.
In both rewrites the task moved from retrieving words to testing a relation between statements - which is what an inference item should do.
3. The trainee defends Q1 on the ground that it is reliable and easy to score. Evaluate that argument, and state what the mentor should tell her.
Read the worked answer
The argument confuses reliability with validity. Q1 will produce the same mark from every examiner - that is reliability - but it yields no evidence about whether the candidate can follow the passage's reasoning, which is the construct the test claims to measure. A test can be flawlessly consistent and still measure the wrong thing.
Three points the mentor should make:
- Blueprint alignment. Ask what proportion of the paper is meant to test literal retrieval versus inference. One literal item out of ten is defensible; one out of three is not, because a candidate who only scans can score a third of the section.
- Marking economy is not the main criterion. Difficulty of writing good distractors is a cost paid once by the item writer; marking ease is a benefit to the examiner. Cheap-to-mark items that measure nothing still cost the school a valid result.
- The same argument applies to Q3, in reverse. 'Higher-order' is not a licence for an unkeyed item. Higher-order items are harder to write, not exempt from having a defensible key.
A practical instruction: keep Q1 only if the blueprint has a literal-detail slot, and replace Q3 either with a keyed best-supported item or move it to the subjective section with a written rubric.