A mean that rises while the pass rate falls: reading Class X Mathematics data
You are the TGT who also convenes the school's result-review group. Class X has 50 students on roll. In Term 1 the Mathematics mean was 48 marks out of 100 with 60 per cent passing (pass mark 33). In Term 2, after you changed to weekly short tests, the Mathematics mean rose to 54 but the pass percentage fell to 52 per cent. Six students were absent for the Term 2 paper; all six had passed in Term 1, four of them with marks between 60 and 72. The principal's note says: "Mean has improved by 6 marks, so the weekly tests worked. Please brief the staff and extend the same method to Science." A colleague adds that the Mathematics teacher whose sections show the lowest pass rate should be moved to junior classes. You have the section-wise mark sheets, the test records and one staff meeting of 20 minutes.
1. Evaluate the claim that the weekly tests worked. Construct at least one pattern of individual mark movements that is consistent with the Term 1 mean, the Term 2 mean and the fall in pass percentage.
Read the worked answer
The claim assumes the mean summarises the whole distribution. Check totals first: Term 1 total = 50 × 48 = 2,400; Term 2 total = 50 × 54 = 2,700. So the aggregate rose by 300 marks while the number passing fell from 30 to 26.
A consistent decomposition:
- 6 students fell from about 36 to about 32 (six losses of 4 = −24), taking six out of the pass band;
- 2 students rose from 30 to 34 (+8), bringing two into the pass band - net pass count 30 − 6 + 2 = 26, i.e. 52 per cent;
- 10 students rose from about 62 to about 90 (+280);
- the remaining 32 students gained roughly 1 mark each (+36). Total change = −24 + 8 + 280 + 36 = +300, giving a mean of 54.
So the mean is being pulled by a small group of large gains at the top while the tail got slightly worse - exactly the group a pass-rate target is about. A rise in mean with a fall in pass rate is a dispersion effect, not evidence of overall improvement. The weekly tests may have widened the gap rather than closed it.
2. How far do the six absentees and the possibility of a harder or easier paper affect the comparison, and what minimum additional data would you bring to the staff meeting?
Read the worked answer
Six absentees out of 50 is 12 per cent of the cohort, so their exclusion is not trivial. Four were scoring 60–72, well above the pass mark. If they had sat Term 2 at similar levels, the pass count would have been at least 30 rather than 26, and the pass percentage would have been close to 60 per cent or better - the headline "pass rate fell" would largely disappear. The Term 2 figure is therefore computed on a different and stronger-sitting population than Term 1, which alone can manufacture the divergence you were asked to explain.
Paper difficulty is the second confound: a mean can move 6 marks simply by shifting item difficulty or mark allocation, with no change in learning. Without a common anchor item across the two papers, the comparison is not a measurement of teaching.
Minimum additional data to bring: (a) the pass rate recomputed on the 44 students who sat both papers; (b) section-wise means and pass counts, not just school means; (c) the mark distribution in bands (0–32, 33–50, 51–70, 71–100) for both terms; (d) three questions repeated verbatim in both papers, compared item-wise; (e) attendance at the weekly tests for the bottom 15 students, since the method may not have reached them at all.
3. The principal wants the lowest-pass-rate teacher moved to junior classes. Give your reasoned response, and state the minimum evidence that would make such a decision defensible.
Read the worked answer
Refuse the inference, not the concern. Three reasons the current data cannot carry that decision.
First, attribution: section pass rates confound teacher with intake. Sections are not randomly assigned; the section with the lowest pass rate may have entered the year with the weakest cohort. You need prior-year scores for the same students to compare like with like.
Second, sample size: a section of 40 with 16 passes versus a section with 20 passes is a difference of four students - well within normal year-to-year variation. A single term is far too short a series to call it a teacher effect.
Third, the measure is wrong: pass rate at a single threshold discards the movement of students at 25 to 32, which is precisely where a TGT's work shows up.
Defensible minimum: two or three years of value-added data (student entry score versus exit score) by section; confirmation that the same students were compared; classroom observation by you or the vice-principal with a written record; and a conversation with the teacher about what is happening in that section. Only then is a support plan - paired teaching, shared test papers, a review of the bottom 15 by name - the proportionate first step, with transfer considered only if the pattern survives all of the above.