The Quiz That Measured the Wrong Thing
Ms. Bhatt teaches seventh-grade science. Her class of 30 includes 9 English learners at ELPAC Levels 2–4. At the end of a unit on the water cycle she gave a 10-item quiz: 6 multiple-choice items (four with diagrams) and 4 short-answer items requiring two to three sentences, all in English. Mean score for non-ELs was 82%; for ELs, 41%. Item analysis: ELs answered 5 of 6 multiple-choice items correctly, including the diagram items, but earned zero of four short-answer items - many left blank. One student wrote a scientifically correct answer in Vietnamese. Ms. Bhatt is considering dropping the quiz, re-teaching the whole unit, or redesigning the assessment. Her principal has told her to "hold ELs to the same standard as everyone else."
1. What is this quiz actually measuring for the EL students? Use the item-level pattern to justify your answer.
Read the worked answer
The two halves of the quiz are measuring different things, and only one of them is science.
- Multiple choice, especially diagram items: ELs are at 5/6. Visuals carry the content load, so the items let students show what they know about the water cycle without routing it through extended English production. These scores are roughly valid for the intended construct.
- Short answer (0/4): a two-to-three-sentence constructed response under time pressure simultaneously demands content knowledge, academic vocabulary, English syntax, and writing fluency. Blank responses and a correct answer written in Vietnamese are the signature of a production barrier, not an empty head.
So the short-answer section carries construct-irrelevant variance: score differences on it reflect English writing proficiency as much as science understanding. The 41% class mean is not a valid estimate of what these students know, and you cannot tell from it whether to re-teach the unit. That is the key point - the data as currently collected do not support either "drop it" or "re-teach everything."
2. The principal says, "hold ELs to the same standard." How do you respond, and where is the line between an accommodation and a modification here?
Read the worked answer
Agree with the premise, redirect the inference. "Same standard" means the same content expectation - explain how water moves through evaporation, condensation, and precipitation and why - not the identical delivery format. Holding students to a standard they cannot demonstrate because of the response format is not rigor; it is a measurement error that gets recorded as a grade.
Accommodations that keep the construct intact: a bilingual glossary of the unit's key terms; sentence frames ("Water evaporates when ___ because ___"); an option to answer orally with the same rubric; extra time; allowing a written response in Vietnamese with translation for scoring against the same content rubric.
Modifications that change the construct: reducing short answers to labeling a diagram only, supplying a word bank that gives away the causal relationship, or dropping the explanation requirement entirely.
The defensible line is: change the route, not the destination. If the standard requires explaining a causal mechanism, every student must explain a causal mechanism; only the medium and scaffolds vary. Note the tension the principal should see: her current quiz already changes the route for non-ELs by letting them use written English, which is itself a medium advantage.
3. Propose three specific changes to the quiz and explain for each why it raises validity rather than lowering the bar.
Read the worked answer
1. Replace two short answers with a labeled diagram plus a required caption using a causal frame. The student still must show the mechanism and its direction (evaporation → condensation → precipitation, and where energy enters). The frame supplies syntax, not content. Validity rises because the item now isolates the science from English sentence construction.
2. Convert one short answer to a structured oral response, scored on the same content rubric. Speaking is a legitimate demonstration of the same standard and is the modality in which several of these students are strongest. Keep the rubric criteria identical so the expectation is visibly unchanged.
3. Keep one full short-answer item as-is, and score it on a two-part rubric: content and language conventions reported separately. This preserves the original demand - so nothing is being hidden or excused - while making the language component visible and separately teachable instead of silently collapsing into the science grade.
What to avoid: translating the whole quiz into Vietnamese for students literate in Vietnamese but not English measures decoding of Vietnamese, not science, and does nothing for the students whose L1 is another language. Selective, purposeful use of L1 is the move, not wholesale translation.