Reading the Item Statistics: A Flawed Question, a Real Error Pattern
After a term of teaching English structure, a teacher gives 44 Form Two students a 40-item multiple-choice test. The class mean is 18.4 with a standard deviation of 5.1. Item analysis shows the following. Q12, on subject-verb agreement with a postponed subject, has a facility of 0.18 and negative discrimination of -0.12. Q17, on present perfect versus simple past, has a facility of 0.44 and discrimination of 0.31. Q24, on article use, has a facility of 0.71 and discrimination of 0.05. Q31, on third-person -s, has a facility of 0.86 and discrimination of 0.42. Meanwhile, 27 of the 44 students write 'He has gone to Accra yesterday', 'I am living here since 2019' and 'She don't like it' in their essays. The teacher wants to reteach the whole tense system for three weeks before the examination and to drop Q12 from the record because 'the students failed it'.
1. Interpret the four item statistics, stating what the teacher may and may not conclude from them. Decide what to do with Q12.
Read the worked answer
What the statistics mean. Facility is the proportion answering correctly; discrimination shows whether high scorers did better than low scorers on that item. They answer different questions.
- Q12: facility 0.18 with discrimination -0.12. An item that the stronger students do worse on is a warning, not a result. Before concluding that the class cannot handle postponed subjects, inspect the wording, the key and the distractor chosen by the top students. The likely causes are ambiguity, a miskey, or a construction on which usage genuinely varies. Deleting the item because students failed is the wrong reason; revising or discarding it after inspection is defensible.
- Q17: facility 0.44, discrimination 0.31. Hard but functioning well. It separates stronger from weaker students and is a genuine teaching target.
- Q24: facility 0.71, discrimination 0.05. Easy and undiscriminating - possibly guessable, or testing something the class already knows. Keep only if the construct matters; otherwise replace.
- Q31: facility 0.86, discrimination 0.42. Easy but still discriminating; it works as a check and should stay.
What may not be concluded. Facility is not a measure of teaching quality, and with 44 students the standard error is large enough that differences of a few percentage points are noise. A difficult item may test a worthwhile structure, and an easy item may test nothing.
Decision on Q12. Retain the score provisionally, re-read the item, count how many high scorers chose each distractor, and rewrite it. Do not drop it simply because it was hard; that removes the evidence of a problem rather than the problem.
2. Distinguish the errors in the essay pattern from mistakes, and say what the pattern indicates about the learners' interlanguage.
Read the worked answer
Error versus mistake. A mistake is a performance slip the learner can correct when attention is drawn to it: a dropped -s in a hurried sentence, a wrong preposition under time pressure. An error is systematic and rule-governed; the learner produces it because their current internal grammar generates it, and pointing it out does not remove it.
Test the pattern. Ask three students to read their own sentences aloud and self-correct. If they say 'I meant has been living' but cannot explain why, and if the same form recurs the next week, these are errors, not mistakes. Frequency supports this: 27 of 44 is a shared system, not individual carelessness.
What the interlanguage shows. The three sentences are not random. 'He has gone to Accra yesterday' shows a present perfect form used where the learner marks past time lexically with 'yesterday' rather than morphologically. 'I am living here since 2019' shows a progressive used to carry a state-plus-duration meaning, with 'since' doing the aspectual work. 'She don't like it' shows the unmarked verb form generalised across persons, with the -s not yet attached to third-person singular. Read together, the learners mark time and person with adverbs and context rather than with verb endings - a coherent strategy, and one that many Ghanaian learners share.
Implication. These are not signs of confusion to be cured by more exposure. They need focused contrastive teaching: the learner must notice where English obliges a formal distinction that their current system handles lexically.
3. Plan a two-week response that is better than reteaching the whole tense system, and explain how you would check whether it worked.
Read the worked answer
Why not reteach everything. Three weeks on the whole tense system spends time on structures the class already controls, buries the three real targets, and leaves no time for production. The evidence points to a narrow set of contrasts, not a system failure.
Two-week plan (four lessons).
- Lesson 1: present perfect versus simple past with time adverbials. Twelve sentence pairs; students sort 'yesterday / last year / in 2019' against 'since / for / already / yet', then rewrite five of their own sentences from a personal timeline.
- Lesson 2: state verbs and the progressive. Focus on 'live, know, want, belong'. Students hunt the error pattern in anonymised scripts and repair it, then write four true sentences about their own lives.
- Lesson 3: third-person -s. Ten-minute drills inside a speaking game, then a short written paragraph about a family member, with the class editing only for that feature.
- Lesson 4: integration. Students revise a previous essay paragraph, editing only for the three targets, and exchange with a partner using a three-item checklist.
Checking whether it worked. Use a five-item parallel test in week one and week three - new sentences, same constructs - and compare item facilities rather than totals. More importantly, look for transfer: collect a new piece of writing two weeks later and count the three target patterns. If the error rate in free writing has fallen, the teaching worked; if only the test score rose, it has been learned for the test and not for use. Keep Q12 out of this judgement until it has been rewritten.