The 8:15 That Nobody Heard: Revising a Listening Item After a Trial
You sit on the item-review panel for the SEC Maltese as a Foreign Language Listening paper. A trial version was sat by 42 Year 10 candidates whose first language is not Maltese. The recording was played twice at about 140 words per minute, with a 30-second pause between playings, and candidates answered four multiple-choice items on a school announcement. Item 4 asks what time students must be in the building. The announcement says the course begins at 8:30, students must be in the building by 8:15, and the library opens at 9:00; the key is 8:15. Twenty-six candidates chose 8:30, nine chose the key, and seven chose 9:00. Item 6 asks for the name of the town where the school is located; the key is 'Ħal-Qormi'. The scripts show 'Ħal-Qormi' (19), 'Ħal Qormi' (12), 'Hal Qormi' (8) and 'Hal-Qormi' (3). The panel meets once for 90 minutes and must sign off the paper for the next session.
1. Item 4 has a facility of 0.21, with 62% of candidates on one distractor. Decide whether to keep, revise or delete the item, justifying your decision with reference to construct, audio conditions and distractor design.
Read the worked answer
Decision: revise, do not delete - but only if 'sal-' is inside the syllabus.
- Construct. The item targets listening for a specific detail plus a deadline marker ('sal-8:15' = by 8:15). That is a legitimate B1 listening skill and the paper needs it, so deleting costs coverage.
- Why it misfired. The key depends on a bound form, 'sal-', and on the candidate suppressing the first and more semantically prominent time (8:30, the course start). At 140 wpm - above the 120–130 wpm normally used at this level - the deadline is heard once and then buried.
- Distractor behaviour. 26 of 42 on a single option is not random guessing; it shows a systematic reading of 'the course begins at 8:30' as the answer to 'when must I be there'. The item is currently a trap, not a comprehension measure.
- Revision. Slow the recording to 125 wpm, state the deadline after the start time, and repeat it once in a second sentence ('...mhux 8:30'). Keep 8:30 as a distractor so the item still discriminates.
- Quality gate. Re-trial with 20 candidates; require facility between 0.40 and 0.80. If facility stays below 0.30, delete and replace with a different detail item.
2. Rule on the four spellings recorded for Item 6, and state a principle the panel can apply to any proper name in the Listening paper.
Read the worked answer
Award the mark for all four forms: 'Ħal-Qormi', 'Ħal Qormi', 'Hal Qormi', 'Hal-Qormi'.
- What is being tested. Listening comprehension of a place name, not Maltese orthography. The sound sequence is unambiguous; a Maltese reader maps every one of those four strings to the same town.
- Why penalising is wrong. Deducting for a missing 'ħ' or a missing hyphen imports the orthography construct from the Writing and Language use papers into a paper that never taught or tested it. It also punishes candidates whose first language has no 'ħ'.
- Principle to adopt. In Listening, accept any spelling that a Maltese reader can map unambiguously to the heard name; diacritics and hyphenation are not penalised. Orthography is assessed where it is the stated construct.
- Exception. If the announcement deliberately spells the name out letter by letter, spelling genuinely is part of the input, exact form is required, and the item should say so. That is not the case here.
- Administrative step. List the accepted variants in the mark scheme so that two markers reach the same decision without conferring.
3. Write the revised Item 4 (stem and four options) and state the key, keeping the same construct.
Read the worked answer
Revised item
Sa liema ħin iridu jkunu fil-bini l-istudenti? A 8:00 B 8:15 C 8:30 D 9:00
Key: B (8:15)
Revised audio line
'Il-kors jibda fit-8:30. Però kulħadd irid ikun fil-bini kmieni - sal-8:15, mhux 8:30. Il-librerija tiftaħ fid-9:00.'
Why this keeps the construct while fixing the item
- The deadline is now stated after the start time, so a candidate who hears only the first number does not automatically succeed.
- 'sal-8:15' is followed by an explicit contrast ('mhux 8:30'), which turns the distractor into a test of comprehension rather than of memory load.
- 8:00 is added as a low-frequency distractor to check that candidates are not simply choosing the earliest time.
- The library time stays as a third plausible option, preserving the original distractor set.
- The stem still asks one thing only: a time limit. No second construct is smuggled in.