The 'Bank Holiday' Problem in a Listening Passage
In the listening section, a British speaker says: 'We'd planned to open on the Monday, but it's a bank holiday, so the suppliers won't be delivering.' Question 9 asks: 'Why will the suppliers not deliver on Monday?' Options: A) They are on strike. B) It is a public holiday. C) The shop is closed. D) Their van has broken down. The key is B. The section is taken by international candidates, and the passage contains no explanation of 'bank holiday'. In the pilot, 61% chose B, 22% chose A, 12% chose C, and 5% chose D. A reviewer argues that the item tests knowledge of British culture rather than listening, because 'bank holiday' is a culture-specific term. Another reviewer argues that 'holiday' and 'won't be delivering' provide enough context for inference. The listening section has 20 items and counts for 25% of the total. The committee must decide whether to keep, revise, or drop the item, and whether the term should be treated as vocabulary or cultural knowledge.
1. Does Item 9 test listening comprehension or cultural knowledge? Decide, and recommend whether to keep, revise, or drop.
Read the worked answer
The item sits on the boundary. 'Bank holiday' is a culture-specific term for a public holiday in the UK; a candidate who does not know it may still infer from 'holiday' and 'suppliers won't be delivering' that normal business is suspended. But 'bank' can mislead candidates into thinking the closure is limited to financial institutions, and option A is a plausible alternative if they interpret the sentence as an industrial dispute. The item therefore tests cultural vocabulary as much as listening. Since the listening section's construct is listening, I would not keep the item in its current form. If the audio can be re-recorded, add an appositive: 'it's a bank holiday-a public holiday-so the suppliers won't be delivering.' That preserves the listening task while removing the cultural barrier. If the form has already been administered, I would drop the item rather than double-key A, because A is not supported by the passage. The term should be listed in the vocabulary syllabus, not treated as assumed cultural knowledge.
2. Rewrite the question so that it assesses inference from context rather than recognition of a cultural term.
Read the worked answer
Rewrite the stem as: 'What does the speaker imply about Monday?' and keep options that test inference: A) The suppliers will work as usual. B) Normal business will not take place. C) The shop will open earlier than planned. D) The suppliers will deliver later that day. Key: B. The speaker says they had planned to open but the day is a bank holiday and the suppliers will not deliver, so the implication is that normal business is suspended. Even a candidate who does not know 'bank holiday' can use 'won't be delivering' and the contrast with 'planned to open' to infer B. This revision shifts the item from lexical recognition to pragmatic inference, which is a legitimate listening skill. If the committee wants to retain the term, it should appear in a separate vocabulary item with a clear context, not as an unglossed obstacle in a listening question. The revised item should be piloted to check that A and C do not attract high-ability candidates.
3. Candidates who have lived in the UK for over a year scored 90% on this item, while recent arrivals scored 40%. What does this suggest, and what is the fairness implication?
Read the worked answer
This is a classic pattern of differential item functioning by cultural exposure. The item is not just measuring listening; it is measuring familiarity with a British institution. Candidates with UK residence have had opportunities to learn 'bank holiday' incidentally, while recent arrivals have not. If the exam is intended to assess language proficiency for admission, this is construct-irrelevant variance and the item is unfair unless the term is taught or explained. The committee should flag the item for DIF review, report the score gap, and avoid using the item in high-stakes decisions without revision. If cultural and contextual knowledge is a legitimate domain, the term can be assessed in that section with a clear rationale and preparation materials. It should not be smuggled into a listening section where the rubric claims to test listening. The fairness implication is that a single unexplained term can change a candidate's total score by 1.25 percentage points, which may matter near a cut score.