Forty Items, One Evening

Go construct by construct, because 'a bad test' is too blunt to act on.

• Content validity (serious). Thirty-one of 40 items are recall. The Integrated Science curriculum and WASSCE both require application, analysis and practical reasoning. The paper therefore samples a narrow slice of the construct and systematically under-rewards students who can reason but do not memorise well. The result is misclassification, not merely a hard test.
• Item 27 (very serious). A mislabelled diagram means the key is not the correct answer. The pattern confirms it: 19 of the top 20 chose the same alternative, and 14 of the weakest chose the key. That is a negative discrimination index, which in a flawed item is diagnostic of a keying error rather than of student weakness. The item must be voided, and its six or seven marks removed from all totals rather than left to punish the able.
• Reliability (moderate to serious). Sixty-two students and 40 binary items is an adequate base, but a single easy-to-write MCQ set with no moderation is vulnerable to guessing and to ambiguous distractors. Recall items usually show weak spread; the 12 to 88 range suggests some items are functioning, but the mean of 41 with heavy recall loading may reflect textbook wording access rather than understanding.
• Essay scoring (serious). Because 47 students gambled their revision on one topic, the essay is measuring topic-guessing, not writing or reasoning. That is a validity threat, and it is predictable when the syllabus scope is known and unconstrained.
• Fairness of the proposed +10 (invalid). A uniform addition changes the mean but not the ranking, and it destroys the criterion-referenced meaning of every mark.

Judgement. Item 27 is a technical error requiring correction. The recall loading and the guessing-driven essay are design faults requiring redesign before the paper can be used for any diagnostic decision.

(a) The ten-mark addition: refuse it, but give him the alternative.

Adding marks does two harmful things. It leaves the ranking unchanged, so it helps nobody who is behind; and it makes every mark meaningless, so the next test has no scale to sit on. It also teaches students that the response to a bad result is a clerical fix, which is precisely the attribution pattern the school says it wants to break.

What he should do instead:

• Void item 27 and recompute the totals out of 34 rather than 40. That is a legitimate correction for a defective item, applied to every student equally, and it is defensible to parents and the HOD.
• Report the corrected mean alongside the original and explain the difference.
• Separate the two decisions: what the marks are for the report card, and what the diagnosis is for teaching. Curving the first while ignoring the second is the error.

(b) The HOD's position: partly right, wrongly used.

The HOD is right that a weak cohort should not be flattered, and right that the WASSCE benchmark is external and unforgiving. But this paper cannot demonstrate what he thinks it demonstrates. Its recall bias means a student could understand science well and still score 41; and the item-27 fault depressed scores for the top group specifically. Using it as evidence that the class is weak is using a flawed ruler to measure a table.

The professional way through: agree that the class needs attention, and disagree about the instrument. Propose a short, moderated re-test of 20 items built against the standards, and let the HOD see both sets of results. If the class is weak, it will show again; if the paper was at fault, the school has learned something. That offers the HOD his conclusion without buying his evidence.

Work backwards from the decisions the marks must inform.

1. Build a table of specifications first. Two axes: content (topics actually taught this term) and cognitive demand. For SHS 2 Integrated Science, something like 30 per cent knowledge and comprehension, 45 per cent application, 25 per cent analysis or practical reasoning. This alone fixes the recall overload before a single item is written.
2. Write items to the grid, then moderate. Each item drafted against a cell of the grid; swap with a colleague who answers it blind. Any item the colleague disputes is rewritten or dropped. Thirty-four good items beat 40 written in an evening.
3. Check how items behave after use. Compute difficulty (proportion correct) and discrimination (D, the proportion correct in the upper group minus the lower group). D of about 0.30 and above is acceptable, 0.10 to 0.30 marginal, negative is a red flag for a keying or ambiguity fault. Review every distractor: an option nobody chooses is dead weight; an option chosen by the strongest students is a warning.
4. Fix the essay. Either publish the scope in advance and let students prepare all three, or offer a choice of two tasks. Under no-choice conditions, test a skills construct (design an investigation; interpret a table) rather than a memorised topic, and mark with a rubric shared with students in advance.
5. Report meaningfully. Give criterion-referenced information - topics mastered, topics not mastered - not just a percentage and a rank. Return the paper, do item-by-item feedback, and retain items in a bank with their statistics.
6. Evaluate the instrument itself. After the next test, compare difficulty and discrimination with this one. That is the action research loop and it is what turns a teacher into someone who can defend a mark.