Designing the External Oral Examination Under Time Pressure
A school runs its Vietnamese (Continuers) oral examination with 18 candidates in one day. Each candidate gets 10 minutes of preparation and a 10-minute interview. The interview has two parts: a conversation about the candidate's in-depth study, then a discussion of a stimulus. The panel notices that candidates who prepare last perform worse, that the stimulus is sometimes exhausted by the first five candidates who share it, and that one examiner tends to ask yes/no questions when time runs short. The school wants results that are comparable across the day and defensible if appealed.
1. Identify the threats to comparability in this schedule and propose fixes that fit the constraints.
Read the worked answer
Threat 1 - Order effects. Later candidates may be disadvantaged by examiner fatigue, by overhearing earlier candidates, or by the school day's cumulative stress. The observed pattern supports this. Fix: rotate examiners across the day so each gets a break; keep candidates in supervised isolation away from the examination room; schedule the strongest and weakest candidates across the day rather than clustering.
Threat 2 - Stimulus exhaustion. If five candidates discuss the same stimulus, the examiner becomes practised and may unconsciously scaffold more, or the stimulus loses freshness. Fix: prepare three or four parallel stimuli of equivalent difficulty and allocate them in rotation, recording which candidate received which. Parallel does not mean identical; it means matched on topic familiarity, vocabulary load, and cognitive demand.
Threat 3 - Examiner drift to yes/no questions. This changes the task for late candidates and lowers the demand. Fix: a written question set for each stimulus, with a rule that at least half the questions require justification or comparison. Brief the examiner that silence is acceptable and that a closed question must be followed by 'Tại sao?' or 'Như thế nào?'.
Threat 4 - Preparation conditions. Confirm all candidates get the same 10 minutes with the same materials and no notes carried in, unless notes are permitted for everyone.
2. How should the panel handle the in-depth study conversation so that it is comparable across candidates with different topics?
Read the worked answer
Problem. Candidates choose different topics, so the content cannot be identical. Comparability must come from the task demands, not the topic.
Solution - a common question architecture. Every candidate is asked the same types of question in the same order:
- Summarise your main finding in one minute.
- Which source was most reliable, and why?
- What would someone who disagrees with you say?
- What would you investigate next?
This standardises cognitive demand: summary, evaluation of evidence, counter-argument, and extension. The examiner may ask follow-ups, but they must stay within these four functions.
Calibration. Before the day, examiners rate two recorded sample conversations against the criteria and agree on bands. During the day, they note the question type asked next to each response so the panel can later check that no candidate received only factual recall questions.
Handling weak topics. If a topic is thin, the questions still work; the candidate's inability to evaluate evidence is the finding, not a reason to rescue them with easier questions.
Recording. Record all interviews (with consent) so an appeal can be checked against what was actually asked.
3. A candidate appeals, claiming that the examiner's questions were harder than those given to a friend earlier in the day. What evidence should the panel examine, and what standard should it apply?
Read the worked answer
Evidence to examine:
- The recordings of both interviews, checked against the common question architecture - did both candidates face summary, evaluation, counter-argument, and extension questions?
- The stimulus allocation record - were the stimuli matched in difficulty?
- The examiner's notes on question types.
- The marking against descriptors, ideally re-marked blind by a second examiner.
Standard to apply. The question is not whether the questions felt harder, but whether the two candidates faced tasks of equivalent demand and were judged by the same standards. Difficulty is a property of the task design, not of the candidate's perception. If both faced the same question functions and matched stimuli, the appeal on grounds of unequal difficulty fails even if the wording differed.
Where the appeal might succeed. If the examiner asked the appellant only closed questions, or used a stimulus that was clearly more abstract, or if the second marking lands more than one band away, the panel should adjust.
Outcome and learning. Record the decision with reasons. If the appeal reveals that the question set was not consistently applied, treat it as a system issue and review all interviews from that session, not just the appellant's.