Nineteen Points: Curving Two Sections and Certifying an Outcome from One Exam
Two instructors teach separate sections of the same statistics course to the same program. Section A (n=40) has a mean of 81, SD 9; Section B (n=42) has a mean of 62, SD 14. The pass mark is 50, and 24% of Section B failed. Section B's instructor used the same textbook but wrote his own exam; Section A reused an item bank that has been in service for six years. The department head proposes adding 19 points to every Section B score so the sections 'match'. Separately, the program certifies PO2 (statistical reasoning) using only this course's final examination, weighted 100% of the grade, and reports the class mean as the attainment figure.
1. Evaluate the curving proposal on measurement grounds, and give the alternative you would put to the department head.
Read the worked answer
The proposal is a norm-referenced fix applied to a criterion-referenced problem. In OBE the pass mark is a standard, and a score is an estimate of distance from that standard. Adding 19 points to every script:
- Changes the meaning of the pass mark. A student who scored 45 has now 'passed' with 64, but no additional competence exists.
- Destroys comparability with every other course and with the outcome claim. Section A's 81 becomes uninterpretable too, since nothing anchors it.
- Is inequitable in both directions - to Section A, whose work is now equivalent to a curve, and to Section B's failing students, who receive a false signal about a prerequisite skill.
- Hides the cause, which is the only thing worth finding.
Alternative: moderation before scoring, not adjustment after.
- Map both exams to the CLOs and compare blueprints - same construct, same cognitive demand?
- Compare item statistics (p-values, discrimination) on the overlap. Section A's six-year-old bank is a validity threat: item exposure and curriculum drift both inflate scores.
- Cross-mark a sample of 15 scripts from each section with the other instructor against a shared rubric.
- If the instruments differ in difficulty, fix the instrument - a common anchor section, or a jointly built exam.
- If the standard is the issue, set it on the instrument before scoring, via a modified Angoff panel, and apply it to both sections.
2. What is wrong with certifying PO2 from this single final exam and reporting the class mean as attainment? Write the attainment statement you would accept instead.
Read the worked answer
Four distinct problems:
- The mean is not an attainment figure. A mean of 62 says nothing about how many students reached the standard. A cohort could have a mean of 70 with 40% below the criterion. Attainment is a count against a criterion, always.
- No per-criterion data. A total score aggregates items that sample different constructs; you cannot attribute any part of it to statistical reasoning specifically.
- Single occasion, single format. One 3-hour sitting, 100% weight: construct underrepresentation plus a decisive penalty for one bad day. Reliability across occasions is unestimated.
- Aggregation across instructors without moderation makes the figure incomparable between sections.
Acceptable statement:
'PO2 (statistical reasoning): of 82 students across two sections, 68 (83%) met the performance standard - a score of 3 or 4 on the statistical-reasoning rubric - on at least two of three assessments: constructed-response items on the final examination, the regression project, and the sampling lab. Instruments were mapped to PO2 and cross-marked on a 20% sample; inter-rater agreement was 87%. Program target: 75%. Met.'
Every element is load-bearing: n, criterion, evidence sources, count, target, and the comparability control.
3. Suppose Section B's low scores might reflect teaching rather than a harder exam. What evidence would distinguish the two explanations, and why does the order of investigation matter?
Read the worked answer
Evidence that discriminates:
- Common anchor items administered to both sections. If Section B underperforms on identical items, the difference is not the exam.
- Item analysis on Section B's exam: uniformly low p-values across easy and hard items suggests teaching or preparation; low p-values concentrated on a few items suggests flawed items or untaught content.
- Prior achievement (incoming GPA, prerequisite grade distribution). If the cohorts differ at entry, neither exam nor teaching alone explains the gap.
- Blind cross-marking of a script sample by the other instructor.
- Classroom observation / materials comparison - coverage of the assessed topics, worked examples, formative checks.
- Item exposure check on Section A. Six years of reuse is long enough for items to circulate; Section A's 81 may itself be inflated, in which case the 'gap' is partly an artefact at the top.
Why order matters: the intervention differs completely depending on the answer. If the instrument is at fault, the correct action is a joint exam and moderation. If teaching is at fault, the correct action is course-level support - and curving would have concealed it for another year. Applying a numerical fix first makes the diagnosis unfalsifiable, because the scores no longer reflect performance. Diagnose, then act; never adjust, then rationalise.