Listening Quiz, Dialect Variation, and Construct Validity

The low scores probably reflect a mix of construct-irrelevant difficulty and genuine listening gaps, and the evidence does not support a single explanation. The students who usually perform well are complaining about phonological features, not about vocabulary or task format. That pattern suggests that the recording’s Caribbean Spanish is unfamiliar to them: aspiration of syllable-final /s/, loss of intervocalic /d/, and /r/→/l/ are regular, describable features, not sloppy speech. If the curriculum has not included those features, the quiz is partly measuring exposure, not comprehension. The chair is right that native speakers talk this way, but that fact does not make this particular quiz fair; a fair assessment can include dialect variation only if students have had opportunities to learn it or if the task does not depend on decoding every reduced form. “Too fast” may be a real rate issue, but it may also be a symptom of unfamiliar reduction. Without item analysis, Mr. Peña cannot know whether the low scores cluster on detail items that require phonetic decoding or on the main-idea item. A curve would hide that information. Reteaching the content would not fix a measurement problem. The first step is to inspect the items and the transcript.

For this grading period, Mr. Peña should not simply curve the scores. He should conduct a quick item analysis: sort items by main idea versus supporting detail, and by whether the answer depends on a reduced phonological form. If the main-idea item was missed by many strong students, that is evidence of a broader listening problem; if only detail items tied to particular reductions were missed, that is evidence of dialect-specific difficulty. He should document the pattern and bring it to the department chair. A defensible adjustment is to drop or reweight the most dialect-dependent items and to offer a retake using a different recording or a transcript-supported version, clearly explained to students. For the future, the department should build a listening assessment blueprint: sample multiple dialects, pre-teach high-frequency phonological processes, use tasks that reward comprehension of meaning rather than phonetic transcription, and include an item-analysis step. The chair’s “native speakers talk like that” argument should be answered with construct validity, not with a claim that Caribbean Spanish is inappropriate. Students need exposure to variation, but assessment should not punish them for what the curriculum has not taught.