Two Teachers, 120 Essays, and a Full Letter Grade of Disagreement
Eleventh-grade English teachers Mr. Ibarra and Ms. Lin each teach three sections of American Literature and share 120 essays from a common rhetorical-analysis prompt. They score with a six-trait analytic rubric (ideas, organization, voice, word choice, sentence fluency, conventions), each trait on a 1-6 scale. When they swap ten papers, Mr. Ibarra's scores average 1.5 points higher on ideas and organization, while Ms. Lin's are 2 points lower on conventions, and she assigns a C to two papers he scored as A. Essay scores are 25 percent of the semester grade, and report cards go home in twelve days. One student, Marcus, has a 504 plan allowing a shortened essay of two body paragraphs instead of three, plus extended time. Another student, Fatima, is at WIDA level 3 and wrote a strong analysis with frequent grammar errors. A parent has already emailed asking why her daughter 'got a B when she worked so hard.'
1. Diagnose why the two teachers' scores diverge. Distinguish a rubric problem, a rater problem, and a task problem, and identify the evidence you would look for.
Read the worked answer
Three different failures produce the same symptom, and they need different fixes.
A rubric problem exists if the descriptors do not distinguish levels. The likely culprit is at the top of the ideas and organization scales: if the 5 and 6 descriptors both say 'clear and well organized,' raters fill the gap with personal preference. A rubric problem also appears if conventions carries more weight in practice than one-sixth of the score, which is what Ms. Lin's two-point convention gap suggests.
A rater problem exists if the teachers apply the rubric differently. Mr. Ibarra may be scoring effort, growth, or the presence of a thesis rather than the quality of analysis (leniency and halo effects); Ms. Lin may be letting surface errors lower her judgment of ideas (horn effect), which is a common and serious pattern with multilingual writers like Fatima.
A task problem exists if the prompt itself is ambiguous or too open, so that the two teachers are in effect rewarding different things. A prompt that says 'analyze the author's rhetoric' without specifying audience or required evidence invites divergent readings of what a strong response looks like.
Evidence to look for: the full score distributions by trait and by teacher; the correlation between conventions and ideas within each teacher's scores (strong negative correlation for Ms. Lin would confirm the horn effect); three or four papers read blind by both teachers with think-alouds about which descriptor they matched; and the prompt itself, checked against whether the descriptors name features the prompt actually asks for. Marcus's and Fatima's papers should be read last, after calibration, because scoring them first tends to anchor the discussion on accommodation rather than criteria.
2. Design a process to resolve all 120 essays and produce defensible grades in twelve days. Include calibration steps, what to do with papers already scored, and how to handle Marcus's and Fatima's essays.
Read the worked answer
Days 1-2: build an anchor set. Select six papers spanning the range, including one with strong analysis and weak conventions. Both teachers score them independently, then discuss trait by trait until they agree. Rewrite the 4, 5, and 6 descriptors for ideas and organization so that each level names a feature that can be pointed to in a text, for example 'explains how the choice affects the audience, with two pieces of textual evidence,' rather than 'insightful.' Decide explicitly how much conventions counts and write it down; one trait out of six means it cannot by itself move an A to a C.
Days 3-9: score blind. Papers are identified by number only. Each teacher scores the papers from the other's sections; any paper with a trait score differing by two or more points, or with a total near a grade boundary, goes to a third read. That is roughly the two dozen papers that actually determine report cards.
Days 10-11: rescore previously graded papers that fall at boundaries, and recompute. Where a recalibration changes a grade, change it; a score produced by an unreliable process is an error, and correcting an error is not the same as renegotiating a grade.
Marcus: score the essay against the rubric as a two-paragraph essay. He should not lose organization or ideas points for a structural feature his 504 plan permits, and the accommodation does not lower the standard for analysis within those paragraphs.
Fatima: separate rhetorical analysis from language control. Score ideas, organization, and evidence on the analysis she actually produced, and score conventions on the conventions trait only. Rosetta-style grammar correction is not the goal, and a lower ideas score because of grammar would be a scoring error, not rigor.
Both teachers should sign the same scoring record so the grades have a shared author.
3. How should the teachers communicate with the parent who emailed, and what should change before next year? Address the ethics of changing grades after the fact.
Read the worked answer
The parent's email deserves a specific, non-defensive reply within two school days. The response should name the six traits, show the rubric with the descriptors, point to specific passages in the daughter's essay with the trait language ('the analysis identifies the appeal but does not yet explain the effect on the audience, which is the difference between a 4 and a 5'), and offer a conversation. Effort should be acknowledged without being substituted for the criteria; if the score changes after recalibration, say so plainly and explain why. Parents accept a well-explained B far more readily than a vague one, and the email itself is a record.
On changing grades after the fact: there are two different acts, and they must not be blurred. Correcting a score produced by an unreliable or miscalibrated process is required, is documented, and is defensible. Changing a score because a parent, coach, or administrator is unhappy is not, and doing it selectively for the loudest family is the real ethics problem. The rule is that the same criteria and the same review process apply to every paper, including the ones nobody complains about.
Before next year: calibrate before the task, not after. Build the anchor set from last year's papers, norm the department in a 45-minute session, revise descriptors so each level is observable, distribute a student-friendly version of the rubric with the prompt, and build in a blind second read for boundary papers. The most expensive mistake this year was not the disagreement; it was discovering it after 120 essays had already been scored.