Recited Introductions: What the Oral Marks Actually Measure
In a Class 9 English classroom in Thrissur, 42 students take a monthly oral assessment. Each student memorises a 10-line self-introduction and recites it in about 40 seconds. The teacher awards marks out of 5 for 'fluency'. In September, 33 students scored 4 or 5. In an unannounced follow-up interaction, only 7 students could answer 'What did you do last Sunday?' in more than one sentence. The teacher has one 40-minute English period daily and no language lab; one smartphone with a recorder is available. The head teacher wants oral marks to count for 10% of the term grade and asks the teacher to justify the assessment design before the examination committee.
1. Analyse the validity and reliability problems in the current oral assessment.
Read the worked answer
Validity: is it measuring the intended construct?
The construct is 'spoken interaction'. The task is memorised monologue. A student can score 5 by reproducing a rehearsed text with no comprehension, no turn-taking and no spontaneous formulation. This is construct under-representation: interaction is missing. The follow-up result (7 of 42) is strong evidence that the scores do not generalise.
Reliability: are the scores consistent?
- A single rater, a single 40-second sample, and one holistic 'fluency' score out of 5 invite halo effects: confident students, or students with pleasant pronunciation, get high marks regardless of grammar or content.
- 'Fluency' is undefined, so the same performance could receive 3 or 5 depending on the day.
- No recording means no moderation or appeal is possible.
Washback: Students learned that oral English equals memorisation; the classroom reward system now works against spontaneous speech.
Authenticity: The task does not resemble any real use of English the students will meet.
The head teacher should be told plainly: 10% of the grade cannot rest on a task that 35 students can pass without being able to answer a simple question.
2. Redesign the oral assessment within the constraints of 42 students, one daily 40-minute period and one smartphone. Give a concrete schedule.
Read the worked answer
Design: paired information-gap tasks, spread over one week, recorded for moderation.
Task types (rotate, 2 minutes per pair)
- A: picture description with one follow-up question from the teacher.
- B: information gap - each partner has half a timetable and must ask questions to complete it.
- C: a 60-second opinion on a familiar topic, with one 'why?' follow-up.
Schedule (six students per day, Monday to Friday, two spill-over slots)
- First 5 minutes: whole class does a silent sustained reading task, so the teacher is free.
- Next 30 minutes: three pairs are recorded, 4 minutes each including transition; the teacher asks one follow-up question per student.
- Last 5 minutes: gather books; the recordings are named by roll number and task.
Over the month: rotate task types so each student is recorded on at least two different task types; use the higher-quality evidence for the term mark and keep both.
Feasibility check: 42 students × 2 minutes of recorded speech equals 84 minutes of audio across a week, easily stored on one phone. The teacher listens to six recordings per day during the lunch break for scoring. This replaces the recitation, not adds to it.
3. Write a rubric for the redesigned task and explain how to make scoring consistent with a colleague.
Read the worked answer
Rubric (4 criteria, 4 bands, total 16 converted to 10)
| Criterion | 4 | 3 | 2 | 1 |
|---|---|---|---|---|
| Communicative effectiveness | Completes the task; listener understands without effort | Mostly completes; occasional strain | Partly completes; frequent strain | Minimal message |
| Interaction | Initiates and answers follow-ups; repairs breakdowns | Answers follow-ups; limited initiation | One-word or formulaic answers | No response or off-task |
| Grammar and range | Several structures, errors do not block meaning | Some variety, noticeable errors | Simple repeated patterns | Fragmented |
| Vocabulary and pronunciation | Adequate words; clear enough to follow | Mostly adequate; some unclear words | Limited words; listener must work | Unintelligible or absent |
Consistency measures
- Score from recordings, not live, so re-listening is possible.
- Double-mark 10 anonymous recordings with a colleague, then discuss any script where the totals differ by 3 or more.
- Build two anchor recordings - one clear 3 and one clear 2 - and use them before every scoring session.
- Score one criterion at a time across all recordings, not one student at a time; this reduces halo.
- Record the date and task type; do not compare a picture description with an opinion task without noting the difference.
This gives the examination committee a defensible 10%: recorded evidence, explicit criteria and documented moderation.