Reading a Coaching Centre's Improvement Claim
A test-prep centre advertises: 'Our students improve by an average of 24 marks - the best in the city.' It publishes results for its 40-student batch: two students improved by 210 and 185 marks respectively; five students improved by 15 marks each; thirty students improved by between 4 and 9 marks; and three students' scores fell, by 12, 9 and 5 marks. A rival centre with a 120-student batch reports a mean improvement of 9 marks, a median of 8 marks and a standard deviation of 6 marks; its dropout rate is 5 %, against 28 % at the first centre, which counts only students who sat the final mock. A student now scoring 420 out of 720 asks for a realistic estimate of her score after six months, and wants to know whether the first centre's claim means it teaches better.
1. Using the first centre's own data, check the 'average of 24 marks' claim and compute a more representative figure.
Read the worked answer
Reconstruct the sum (taking the 30 mid-range students at ≈6.5 marks, the midpoint of 4–9).
| Group | Contribution |
|---|---|
| 210 + 185 | 395 |
| 5 students × 15 | 75 |
| 30 students × 6.5 | 195 |
| 3 students: −12, −9, −5 | −26 |
| Total | 639 |
Mean = 639/40 = ≈16.0 marks, not 24. Even if the thirty are all taken at the top of their range (9 marks), the total becomes 639 + 75 = 714, giving a mean of 17.9 - still nowhere near 24. The advertised figure cannot be derived from the published data, so either a different (undisclosed) group or a different statistic was used.
Median. With 40 values, the median is the average of the 20th and 21st in order. The three falls occupy positions 1–3, so positions 4–33 are the low improvers; both the 20th and 21st values therefore lie in the 4–9 band, giving a median of roughly 6–7 marks.
Judgement. The mean (≈16) is more than double the median (≈6.5) because two students contribute 395 of the 639 total - 62 % of all the improvement in the batch. The honest headline is 'a typical student gains about 6–7 marks', with the caveat that two exceptional students pulled the average up.
2. Give the student a realistic estimate of her score after six months at the first centre, and say which numbers from the rival centre change your advice.
Read the worked answer
Estimate for the first centre. A student at 420/720 is not one of the two outliers, so the median is the right predictor, not the mean: 420 + 6.5 ≈ 426–427 out of 720, with a realistic band of about 424–429.
Why not use the mean. The mean of 16 would predict 436, but that figure depends entirely on two students who gained 210 and 185 marks - probably atypical cases (a repeat student, or someone who barely attempted the first diagnostic). A prediction built on n = 2 is not a prediction.
What the rival's numbers add.
- Mean 9 and median 8 are close together, so there are no dominating outliers: the typical and the average student gain about the same.
- SD 6 means roughly two-thirds of its students gain between 3 and 15 marks - a tighter, more predictable band, and its central value (8) is higher than the first centre's median (6.5).
- Dropout 5 % versus 28 %: the first centre reports only students who sat the final mock, so its weakest or most discouraged students are missing, which flatters its figures. That is survivorship bias, and it is enough on its own to make a head-to-head comparison unfair.
Advice. Expect around 428–432 at the rival centre - a modest but more dependable gain - and treat the first centre's '24 marks' as marketing rather than a forecast.
3. The first centre's 'best in the city' claim rests on comparing its mean with the rival's. List the information you would need before accepting any such comparison.
Read the worked answer
A mean-vs-mean comparison is only fair if the two groups are alike and the measurement is alike. I would ask for:
- Baseline scores. A batch starting at 300 has far more room to gain than one starting at 550. Compare gain-per-student only after matching starting points, or compare final scores, not gains.
- Who is counted. Both centres should report on everyone who enrolled, not only those who sat the final test. The first centre's 28 % dropout almost certainly inflates its mean.
- Same instrument at both ends. If the first centre uses its own easier mock as the post-test, its gains are not comparable with a standardised test.
- Sample size and spread. 40 versus 120 students: the smaller group's mean is far less stable, and a single outlier moves it by 5 marks. Standard deviations or confidence intervals should be quoted.
- Selection effects. If the first centre admits only motivated repeaters, or offers fee refunds for attendance, its 'gain' partly reflects who joined rather than what it taught.
- Definition of 'improvement'. Best-of-three attempts, practice-effect from repeated papers, and self-reported scores all inflate the result.
Judgement. None of this makes the first centre dishonest - it makes the claim uninterpretable. The defensible statement is 'median gain of about 6–7 marks among students who completed the course'; 'best in the city' needs matched groups, complete follow-up and a common test, which the published data do not provide.