Scoring a Radio Interview with Overlapping Speech
You are on a committee piloting a listening-comprehension section for the Mother Tongue Skolt Sámi exam. The stimulus is a 4-minute community-radio interview: the host speaks Skolt Sámi, and the guest, an 82-year-old fisherman, switches to Finnish for one technical term about boat registration. About 40 seconds of the recording contain overlapping boat-engine noise. Of six draft items, item 4 asks how many permits the guest's father had; the transcript confirms two, but the number is masked by noise and two pilot students heard three. Item 5 asks the guest's attitude toward the new registration rule; he uses a hedge and rising intonation, and the bilingual Finnish-Skolt student answers it correctly while the heritage learner does not. The committee chair proposes keeping item 4 because the transcript is authoritative and dropping item 5 as subjective. You have the audio log, transcript, and pilot responses.
1. Should item 4 be kept, revised, or dropped? Justify using listening-assessment principles.
Read the worked answer
Item 4 should be dropped or the audio should be repaired. A listening item is valid only if the tested detail is reliably audible to the intended candidates. The transcript is a researcher's record, not evidence that learners can hear two; if the engine noise masks the number and pilot students hear three, the item is measuring recording quality or lucky guessing. Re-recording, cleaning the segment, or replacing the item with an audible detail are the defensible options. Keeping it because the transcript is authoritative inverts the construct: it turns a listening test into a transcript-reading test. If the number matters, repeat it in a clear follow-up question in the interview.
2. Is item 5 defensible? If yes, rewrite it; if no, explain what evidence would be needed.
Read the worked answer
Item 5 can be defensible if the hedge and intonation are audible and the item asks for evidence. Rewrite it as: 'Which phrase best shows how the guest feels about the new rule?' with options that quote or paraphrase his hedge. The fact that one bilingual student answered and one heritage learner did not is not proof of bias; the sample is too small. But the committee should check whether the cue depends on Finnish-Skolt code-switching or on a prosodic pattern that all learners can hear. If it does, pre-teach or replace. If it does not, keep the item and ask for the supporting phrase.
3. What accommodation, if any, should be made for the Finnish code-switch, and why?
Read the worked answer
Pre-teach the Finnish technical term in the instructions, or gloss it once in the item booklet. The interview's main point is the registration rule, not knowledge of Finnish vocabulary. A short gloss such as rekisteri = registration record keeps the listening construct intact while removing an irrelevant language barrier. Do not translate the whole segment: that would make the task a reading task. If the code-switch itself is the point, ask a separate item about why the guest switches, and make sure the reason is inferable from the host's response.