Is this recording fit to assess listening?
Porirua College has 14 students in its Level 1 Gagana Tokelau class. Their teacher, Ms Tovio, wants to use a 2 minute 40 second phone recording as the listening assessment. In it, a Pulenuku on Atafu announces a village clean-up: which groups work on which days, what to bring, and a changed start time. The recording was made outdoors; wind buffets the microphone, the speaker moves quickly between points, and several words in the announcement are formal ones the class has never met. Three students arrived from Fakaofo eighteen months ago and say the speaker 'sounds like Atafu'. Two students were born in Wellington; they cope with everyday conversation but not with announcements. The assessment is in ten school days. Ms Tovio must decide, before assessment day, whether students hear the recording once or twice, and apply the same rule to everyone. The head of department asks whether the recording is fit for purpose, and Ms Tovio also has to decide how to mark a student whose answers are correct but written mostly in English.
1. Identify the two biggest threats to the validity of this recording as an assessment and explain what Ms Tovio should change before assessment day.
Read the worked answer
The two threats are construct-irrelevant difficulty and inconsistent conditions.
- Wind and speed are not what the standard measures. The standard asks whether a student can understand spoken Tokelauan in an authentic context; it does not ask whether they can hear through a buffeting microphone. If noise masks word endings, a student who would understand the announcement in a quiet room fails for a reason unrelated to comprehension. Fix: re-record or clean the audio, or play it in two shorter segments. Keep the fast delivery - speed is a genuine listening demand - but remove the masking.
- Unknown formal vocabulary plus a single play. Several words are new to all 14 students. If those words are the point, they must be taught or glossed beforehand; if the point is gist and detail, a five-item pre-teach protects validity without giving the answers away.
- Consistency. Decide the play rule now, put it on the task sheet, and apply it to everyone, including anyone absent on the day. A rule invented after seeing who struggled cannot be defended to a moderator.
The test is not 'is this authentic?' - it is authentic - but 'does this task let students show comprehension rather than microphone tolerance?'
2. The Fakaofo students say the speaker 'sounds like Atafu'. How should this affect planning and marking?
Read the worked answer
Treat it as information about the task, not about the students.
Tokelauan has regional variation, and the class's audio resources use one variety; a speaker from another atoll will differ in vowels, stress and some vocabulary. The students are not making errors by noticing this, and they should not be marked down for using Fakaofo forms in their own answers - those forms are legitimate Tokelauan, and the standard assesses comprehension, not conformity to one atoll.
The practical question is whether the gap is lexical or phonological:
- If they missed content because of a word used differently in their variety, pre-teach that word to the whole class. That is a fair fix and it benefits everyone.
- If it is only accent, give them two or three minutes of this same speaker before assessment day so the voice is familiar. Exposure, not translation.
Do not ask them to render the announcement in their own variety as a check - that tests translation, not listening. If their answers show they followed the groups, days, items and time change, they have met the standard.
3. A student answers all six questions correctly but writes in English with three Tokelauan words. How should Ms Tovio handle this?
Read the worked answer
Check the assessment conditions first, then mark what is actually being assessed.
- If the standard permits responses in English or Tokelauan (common where the standard assesses comprehension, not writing), mark the content. Did the student identify the groups, the days, what to bring and the changed time? The three Tokelauan words are a bonus, not a requirement, and the English is not a deficiency.
- If the standard requires the response in Tokelauan, this is a format problem, not a comprehension problem. Record the comprehension evidence, then give the student a short opportunity to give the same answers orally in Tokelauan rather than re-teaching the content.
Either way, the judgement principle is the same: the response language must not be allowed to hide what the student knows, and it must not be used to inflate a script that shows no comprehension. A fluent English answer with no specific detail from the announcement still fails; a plain answer with the four correct details still passes.