The Pass Rate That Rose While Both Centres Fell
A committee reviewing learner performance at a state open university has produced a two-year comparison of two regional centres. Centre A: Year 1, 900 learners appeared and 540 passed; Year 2, 200 appeared and 110 passed. Centre B: Year 1, 100 appeared and 90 passed; Year 2, 800 appeared and 680 passed. Total appearances were 1,000 in both years. The committee's draft note says: 'The overall pass percentage has risen from 63% to 79% in one year, and Centre B in particular has shown remarkable growth in the number of successful learners. Teaching quality across the university has therefore improved, and the model followed at Centre B should be extended to all centres.' A member objects, but the note is scheduled for adoption. You are asked to comment in writing before adoption. Enrolments were 1,500 and 1,300 at Centre A, and 300 and 1,100 at Centre B - appearances are lower than enrolments in every case.
1. Using the figures, test the committee's inference that teaching quality improved everywhere. Show the computations and name the statistical phenomenon at work.
Read the worked answer
Centre-wise first, then pooled.
- Centre A: 540/900 = 60.0% → 110/200 = 55.0% (down 5 points)
- Centre B: 90/100 = 90.0% → 680/800 = 85.0% (down 5 points)
- University: 630/1,000 = 63.0% → 790/1,000 = 79.0% (up 16 points)
Both components fell by the same amount, yet the pooled figure rose by 16 points. That is Simpson's paradox: the pooled percentage is a weighted average, and the weights moved. Centre B's share of appearances rose from 100/1,000 = 10% to 800/1,000 = 80%, so the aggregate is now dominated by the higher-performing centre.
The arithmetic point that settles the argument: a 16-point rise in the pooled rate is fully compatible with a 5-point fall at every single centre. Nothing about 'teaching quality across the university' can be inferred from the pooled number, because the improvement is an artefact of composition, not of performance. The correct summary of the data as given is that pass rates fell at both centres.
2. The note also says Centre B 'has shown remarkable growth'. What is the correct reading, and what further data would you demand before endorsing any transfer of the Centre B model?
Read the worked answer
Growth in volume, not in performance. Centre B's successful learners rose from 90 to 680, an eight-fold increase, but appearances rose eight-fold too (100 to 800) while the pass rate fell from 90% to 85%. The growth is an intake effect: enrolments at B went from 300 to 1,100. Congratulating B on 'remarkable growth' confuses a bigger denominator with a better result, and it happens to be the very confusion that produces the misleading pooled figure.
Data to demand before any transfer.
- Appearances as a share of enrolments, centre-wise and year-wise: Centre A's appearance rate fell sharply (900/1,500 = 60% to 200/1,300 = 15%), which will bias any pass percentage computed on appearances. Was there a change in examination scheduling, or were weak learners filtered out before appearing?
- Intake profile: prior qualifications, age, region, medium of instruction, and whether B's new learners are a different population from A's.
- Course or subject mix, if the two centres serve different programmes.
- Tutor cadre: numbers, qualifications, turnover and the tutor-to-learner ratio in each year.
- Paper equivalence and marking: was the same question paper used, and were scripts marked to the same scheme across centres?
- Unfair-means cases and examination-centre practices.
Without these, transferring the 'B model' would be transferring a composition effect into four other centres.
3. Write the amendment you would move (not more than 100 words) and one performance metric you would substitute for the overall pass percentage.
Read the worked answer
Amendment to be moved.
'The note is withdrawn as drafted. Pass percentages fell at both centres - Centre A from 60.0% to 55.0%, Centre B from 90.0% to 85.0% - while the pooled figure rose from 63.0% to 79.0% only because Centre B's share of appearances rose from 10% to 80%. Growth in Centre B's successful learners, from 90 to 680, reflects an eight-fold rise in appearances, not improved performance. The committee should obtain centre-wise analyses on a consistent denominator, including learners who did not appear, before drawing conclusions or recommending transfer of any model.'
Roughly 95 words, and it does two jobs: it corrects the arithmetic and it blocks the recommendation without asserting the opposite (that quality fell), which the data also do not establish.
Substitute metric. Pass rate per 100 enrolled learners, disaggregated by centre and reported with a confidence interval, ideally on a cohort basis (of those admitted in a given cycle, how many completed and passed within the maximum period). This fixes both defects at once: the denominator is stable and not manipulable by non-appearance, and disaggregation prevents composition shifts from masquerading as improvement.