Moderation Flag: The Writing Task That Everyone Aced
A teacher submits a moderation sample for the external examination preparation unit. The internal writing task asked students to write 120 words about their ideal holiday. Scores were unusually high: 15 of 18 students scored 4 or 5 out of 5 on Accuracy, and the class average was 4.4. The moderation panel notices that the task prompt was given to students two weeks in advance, that they were allowed to use dictionaries and draft at home, and that the rubric's Accuracy descriptor for a 5 mentions 'a wide range of structures with few errors'. The teacher defends the results, saying the class worked hard and the writing was genuinely good. The panel must decide whether to adjust the scores or query the task design.
1. Diagnose why the score distribution is suspicious, using assessment principles.
Read the worked answer
A class average of 4.4 out of 5 with 15 of 18 students at 4 or 5 is a strong signal of a ceiling effect, meaning the task did not discriminate between students. When almost everyone scores near the top, the task provides little information about relative achievement.
The likely cause is task conditions rather than genuine excellence. Two weeks' advance notice, dictionary access, and home drafting allow students to produce polished text that may not reflect independent writing ability. If the rubric's 5 descriptor refers to a wide range of structures with few errors, drafted and dictionary-assisted work can meet it without the student being able to do so unaided.
The principle at stake is validity: the task may not be measuring the intended skill (independent written production) under the intended conditions. High scores alone are not proof of a problem, but the combination of ceiling effect and generous conditions is a red flag.
2. Should the panel adjust the scores or query the task design? Recommend a course of action with justification.
Read the worked answer
The panel should query the task design first and adjust scores only if evidence shows the rubric was misapplied. Adjusting scores without changing conditions would penalise students for following the rules they were given, which is unfair.
The recommended action: (1) review the task conditions against the program's requirements for independent work; (2) sample several scripts at each score level to check whether the Accuracy descriptor was applied consistently; (3) if the conditions were too generous, revise the task for future cohorts (for example, in-class writing with limited dictionary use) rather than retroactively penalising this cohort; (4) if the rubric was applied too leniently, recalibrate the sample scores and document the rationale.
The distinction matters: a design flaw calls for redesign, a marking flaw calls for re-marking. The panel should determine which it is before acting.
3. The teacher argues the writing was 'genuinely good'. How should the panel respond, and what evidence would settle the question?
Read the worked answer
The panel should acknowledge that the writing may indeed be good while explaining that quality of the final product is not the only question; the panel must ask whether the product reflects the student's independent ability under comparable conditions. Both can be true: excellent drafted work that overstates unaided proficiency.
Evidence that would settle the question: compare the same students' in-class, timed writing on a similar prompt without dictionaries. If scores drop sharply, the home-drafted task was not measuring the same skill. Also examine whether lower-scoring students' scripts show errors consistent with independent work while top scripts show dictionary-level vocabulary and structures beyond the students' oral or timed-writing performance.
A practical resolution is to retain the task but add an in-class timed piece for moderation comparison, and to recalibrate the rubric descriptors so that 'few errors' is interpreted in light of the conditions under which the work was produced.