A Three-Point Gap in Ideas
Two teachers score the same 30 narrative essays on a six-point analytic rubric with Ideas, Organization, Voice, Word Choice, and Conventions. On essay #12 they disagree by three points in Ideas. Teacher A gives a 5: the essay opens with a striking image of a flooded basement and holds a consistent mood. Teacher B gives a 2: "nothing happens - there's no conflict." The essay runs about 700 words, mostly description of the basement and the family's belongings, with one closing paragraph of reflection. Across the set, the two teachers differ by three or more points on six of the 30 essays, and four of those discrepancies are in Ideas. Scores are due to the district Friday.
1. What is the likely source of the disagreement, and what should the teachers do before Friday?
Read the worked answer
The pattern - clustered in one trait, at a three-point spread - points to an underspecified rubric rather than to careless scoring. The Ideas descriptor is probably doing double duty, covering focus, development, and support, so one teacher weights vivid, controlled description and the other weights the presence of a conflict and a turn. Both readings are consistent with the essay; the rubric just doesn't say which one governs.
The immediate move is calibration, not compromise. Each teacher rereads the descriptor aloud, then independently rescores three of the six disputed essays against the actual language. Where they still differ, they locate the exact phrase they are interpreting differently and write a one-line ruling: for example, "Ideas includes development of a change or realization; sustained description alone caps at 4 unless the reflection supplies a turn." That ruling goes in the team's scoring notes and is applied to all 30, not just the disputed six. Splitting the difference arithmetically produces a number nobody believes.
2. How should the team restructure the rubric and scoring process to prevent this next time?
Read the worked answer
Separate the constructs and build in anchors:
- Split Ideas into "focus and central idea" and "development," so a beautifully rendered scene and an undeveloped narrative no longer have to be scored on the same scale.
- Select anchor papers at scores 2, 3, 4, and 5, including a description-heavy essay, so scorers have a shared referent for exactly the case that divided them.
- Run a short norming session each round: score two papers in common, compare, argue, agree, then score independently.
- Double-score a sample (even 10%) and report agreement; if a trait keeps producing three-point gaps, that trait's descriptor is the problem.
Reliability here is a property of the instrument and the training, not of individual teacher diligence, and the pattern in the data says so.
3. What feedback should essay #12's author receive, given the disagreement?
Read the worked answer
The student should not be told there was a fight over the score, and the student should not receive an averaged number with no explanation. The feedback should name what the essay does well - the flooded-basement image and the sustained mood are real strengths, and both teachers saw them - and then name the one revision that would resolve the disagreement: the piece currently renders a scene but does not stage a change. The closing reflection gestures at one; it could be earned earlier by a concrete turn, such as a discovery, a loss, or a decision in the middle of the scene.
A useful framing for the student: "Your description is strong enough that a reader wants something to happen in it." Where the rubric is genuinely ambiguous, the teacher should say so in the comment rather than pretend the score came down from nowhere; students learn more from an honest statement of what the rubric rewards than from a confident number.