Bimodal Scores on a Unit Test: What the Data Do and Do Not Support
Mr. Alvarez gives a 30-item multiple-choice test at the end of an eleventh-grade U.S. history unit. The class of 28 scores a mean of 71, but the distribution is bimodal: a cluster near 62 and another near 88, with almost nothing between. An item analysis shows 24 of the 30 items ask students to identify a term, date, or person; 6 ask them to interpret a short excerpt. The four English learners and two students with reading disabilities scored between 48 and 58. Every item stem is longer than 40 words. The unit's stated objectives were to explain causes of a policy shift and to evaluate two competing interpretations.
1. What can and cannot be concluded from these scores? Address the bimodal shape, the item analysis, and the small sample.
Read the worked answer
What the data support.
- Construct mismatch. The unit's objectives ask students to explain causes and evaluate interpretations. Three-quarters of the items ask for identification. The test is measuring recall, so a low score is not evidence that students cannot do what the unit targeted. This is the most important finding, and it is separate from any question about student ability.
- A language-load problem worth investigating. Every stem exceeds 40 words, and the six lowest scores sit with the English learners and students with reading disabilities. That pattern is consistent with reading demand interfering with what the test claims to measure. It is not proof - six students is a small group and could reflect other factors - but it warrants checking items individually.
What the data do not support.
- An inference about two kinds of learners. A bimodal distribution with 28 students can be produced by many things: the previous unit's teacher, a review session half the class attended, or simply a gap in the score distribution. With n = 28, one or two students can create an apparent second cluster.
- Fine-grained item conclusions. Per-item statistics on 28 students are unstable; a single item with 60% correct could swing widely on a retest.
- Any conclusion about the four who scored in the 40s beyond "investigate."
Before drawing conclusions, check what distinguishes the clusters - attendance, prior course, review-session participation.
2. Propose revisions to the assessment, and explain how you would keep the revised version from becoming either easier or narrower.
Read the worked answer
Revisions.
- Rebuild the item set to the objectives. At least half the items should require explaining a cause or comparing two interpretations, using short document excerpts. Keep a small number of identification items as a prerequisite check.
- Cut stem length without cutting the task. Rewrite 40-plus-word stems into a short setup plus a short question. Reading load drops; historical reasoning demand stays.
- Separate the reading barrier from the construct. Offer a glossary for genuinely unfamiliar proper nouns, and read-aloud or extended time as the IEPs and 504s require. Do not simplify the reasoning required.
- Score the constructed-response portion with a rubric that names what "explains a cause" looks like at each level, so the grade is not a single multiple-choice number.
How to keep it honest.
- Write the objective-to-item map before writing items. If an objective has no item, the test is narrower than the unit; if an item has no objective, it is testing something you did not teach.
- Field-test new items on a spare class or keep them as ungraded warm-ups and check the distractor patterns.
- Compare the revised test to the old one on the same content: if scores rise and the response quality on cause-and-effect questions rises with them, the test changed what it measured. If only the multiple-choice scores rise, it got easier.
3. A colleague says the fix is to curve the test so the lower cluster is not penalized. Evaluate this against the assessment's purpose.
Read the worked answer
A curve adjusts the reporting; it does not fix what was measured.
The test asked mostly for identification when the unit asked for explanation. Adding points moves every score up and leaves that mismatch intact. Students in the lower cluster would receive a grade that says they met objectives the test never assessed, and students in the upper cluster would receive credit for a skill that was not tested either. The score becomes less informative for everyone.
Where the colleague is right: the current grades are not defensible as evidence of the stated objectives, and Mr. Alvarez should not simply record them and move on. That is a real problem.
Better uses of the same impulse:
- Weight the test as one part of the unit grade rather than the unit grade, and include a performance task aligned to the objectives.
- Re-teach and reassess the interpretive objectives with a short constructed-response task, and let the higher of the two count.
- Report the multiple-choice score for what it is - a recall check - and use it diagnostically.
A curve applied to a misaligned test makes the numbers look fairer while reducing the information available to the students who most need it. The colleague's concern is legitimate; the instrument is the problem.