Listening Test with Caribbean Spanish: Validity and Fairness
A high school Spanish 3 team built a 30-item interpretive listening test. Twelve items were recorded by a Puerto Rican speaker, eight by a Colombian Andean speaker, and ten by a Peninsular speaker. The Puerto Rican recordings include aspiration of /s/, elision of /d/ in -ado, and /r/→/l/ in syllable-final position. After the first administration to 142 students, the department chair notes that nine students who studied in Spain scored highest; 23 heritage speakers of Mexican descent scored lower on Caribbean items than on Andean items; four students with IEPs have extended time. A parent claims the test is 'not real Spanish.' The test counts 20% of the quarter grade. Teachers must decide whether to keep, re-record, or revise before the retake in two weeks.
1. Is the Caribbean accent on the listening test a construct-irrelevant barrier or a legitimate part of Spanish listening proficiency? Decide and justify for this test.
Read the worked answer
It is legitimate in principle but not in this test's current design. Spanish listening proficiency includes understanding major educated varieties, and Caribbean Spanish is not 'bad Spanish'; features such as /s/ aspiration, /d/ elision in -ado, and syllable-final /r/→/l/ are systematic and widely attested. A test that only uses Peninsular or Andean recordings would underrepresent the Spanish-speaking world.
However, the current 30-item test has 12 Caribbean items, 8 Andean, and 10 Peninsular, and the Caribbean items are concentrated in the most difficult section. If students' lower scores on those items reflect unfamiliarity with the variety rather than failure to understand the message, the items are measuring exposure, not proficiency. The parent's claim that it is 'not real Spanish' is wrong, but the test design still needs evidence. I would keep some Caribbean items, check item difficulty and discrimination, and make sure the test overall samples varieties more evenly. The decisive question is whether the items can be answered from context and whether students have had comparable exposure.
2. Propose specific revisions before the retake in two weeks. Include recording, item, and scoring changes.
Read the worked answer
- Re-record the 12 Caribbean items with the same speaker but slow the delivery slightly and reduce background noise; do not erase the features, but do not stack multiple features in one short item.
- Add a short pre-test exposure task: a three-minute ungraded clip with the same speaker and a transcript, so the test measures comprehension after familiarization, not first-contact shock.
- For four of the 12 items, revise the question so the answer depends on an explicit meaning cue rather than a single reduced segment; for example, ask about the reason for a change, not the exact word.
- Balance the blueprint: 10 Caribbean, 10 Andean, 10 Peninsular, with similar item types; do not put all Caribbean items in the last section.
- Keep the same scoring weight but report subscores by variety for diagnostic purposes only; do not lower the standard.
- Give the four IEP students the already-approved extended time and a quiet room; do not change the audio for them only, which would create a different construct.
The retake should be a parallel form, not the same items, if possible.
3. How would you explain the decision to the parent who says the test is 'not real Spanish' and to the 23 heritage speakers who scored lower on Caribbean items?
Read the worked answer
To the parent: 'The recording uses a Puerto Rican speaker. The features you may hear-aspirated s, dropped d in -ado, and r/l variation-are normal in educated Puerto Rican Spanish. They are not errors. The test is not replacing standard Spanish; it is sampling varieties that students may encounter. We are revising the test to make the items fairer, not to remove the variety.' Avoid calling the parent's variety 'standard' and the other 'accent'; explain that all varieties have norms.
To the heritage speakers: 'Your Mexican Spanish is not being graded as wrong. The Caribbean items were harder because you have had less exposure to that variety, and that is a test-design issue as well as a learning opportunity. Here is a 10-minute practice clip; we will review the features and then retake a parallel form. Your overall score will include the same variety balance for everyone.'
The reasoning: validate the linguistic system, separate exposure from proficiency, and show a concrete instructional response. Do not tell heritage speakers they 'just need to listen more' without changing the test.