Two Raters, One Band Apart
A B1 writing paper asks for an email of 120–150 words covering three points: apologise for a missed meeting, propose a new date, and suggest what to do about the postponed agenda item. A candidate's script runs to 168 words, is clearly paragraphed, and covers all three points so that no reader could misread the message. The errors are systematic: seven missing or wrong articles, four uses of present perfect where past simple is required, one word-order slip. Rater A awards B1 because the message is fully effective; Rater B awards A2 because of the error density. The exam double-marks 10% of scripts and resolves disagreements by adjudication. You are the adjudicator.
1. Which band do you award, and which parts of the evidence decide it?
Read the worked answer
B1 - but on the whole script, not on a word-level error count.
- Task fulfilment is complete. All three communicative points are present and a reader would know exactly what to do. The A2 frame is built around short, simple messages; a 168-word paragraphed email handling three separate functions sits outside it.
- The errors are systematic, not random. Missing articles and present-perfect-for-past are the classic interlanguage profile at the B1 boundary - the errors of a learner operating a wider repertoire, not of one whose repertoire is still narrow.
- No error touches the message. The date is unambiguous and the apology and the agenda suggestion are both intelligible on a first reading.
Rater B is counting rather than judging. Error density matters only where it interferes with communication; had the wrong tense obscured when the new date was, the verdict would flip to A2. Record that reasoning in the adjudication note - the next rater needs the principle, not just the band.
2. What does this disagreement reveal about the scale or the rater training?
Read the worked answer
It is a scale problem more than a rater problem. Rater B is applying an accuracy-only reading, and the scale evidently permits that reading; if it had stated that accuracy is one criterion among several and that downgrading requires an effect on communication, both raters would have converged.
Concrete fixes:
- Separate criteria for task fulfilment, coherence, range and accuracy, each scored before an overall band is awarded, so no single criterion can swallow the judgement.
- Written guidance with worked examples of systematic, non-impeding error at the A2/B1 boundary.
- Benchmark scripts re-rated at the start of every session, including one borderline script like this one.
- Log the overturns. Keep the reason for each adjudication; a run of the same reason is a training need, not a series of accidents.
3. The script is 168 words against a 120–150 limit. Does that affect the level awarded?
Read the worked answer
No. Length is not a level criterion, and a word range is a task instruction rather than a descriptor. Ask instead what the extra 18 words are doing. If they add relevant content, ignore them. If they are padding or repetition the candidate needed in order to reach the minimum, note it under task fulfilment, not under language.
The one real concern is bias: an over-long script gives the rater more opportunities to find errors, so length can quietly influence accuracy judgements in either direction. Raters should be instructed to read the whole text for overall effect and to base accuracy on the pattern of error, not on a tally. If the examination board genuinely wants length controlled, it should state a tolerance and a consequence in advance - for example, no penalty up to 200 words, content beyond that not credited - rather than leave it to rater discretion.