Two Shifts, One Cut-off
A state eligibility test is administered in two shifts because 68,400 candidates cannot be accommodated in one sitting. Shift A (41,200 candidates) has a mean raw score of 58.4 with standard deviation 12.1; Shift B (27,200 candidates) has a mean of 64.9 with standard deviation 10.8. The board fixes a single raw cut-off of 80 marks to fill the 6 per cent qualifying band. The outcome: 3.4 per cent of Shift A (about 1,400 candidates) and 7.6 per cent of Shift B (about 2,070) qualify. A Shift A candidate writes a representation arguing that the paper was harder and that the single cut-off is arbitrary. The answer-key challenge window has closed, there is no time or budget for a re-test, and results must be declared in ten days. State precedent requires that selection be 'non-arbitrary and demonstrably rational', and the board's ordinance is silent on shift-wise normalisation.
1. Is the candidate's grievance best characterised as a complaint about difficulty or about comparability? Support your answer with the statistics given.
Read the worked answer
It is a comparability complaint, though it is expressed as a difficulty complaint - and the distinction decides the remedy.
Standardise the cut-off within each shift. In Shift A, z = (80 − 58.4) / 12.1 = 1.79, so a candidate needed to be about 1.8 standard deviations above the shift mean. In Shift B, z = (80 − 64.9) / 10.8 = 1.40. The same raw mark of 80 therefore sits at very different positions in the two distributions, and the tail proportions follow: 3.4 per cent versus 7.6 per cent.
The standard deviations are close (12.1 and 10.8), so the disparity is driven almost entirely by the 6.5-mark gap in means, not by different spreads. If the shift assignment was effectively random across centres, the two candidate pools were comparable in ability before the test; a 6.5-mark difference in outcome is then most plausibly a paper-difficulty (or scoring) effect. Once that is accepted, the raw cut-off no longer compares like with like: two candidates of equal ability receive different verdicts depending on which shift they happened to sit. That is precisely the arbitrariness the precedent forbids. Note also that the board's cut-off currently produces 3,489 qualifiers, about 5.1 per cent, not the intended 6 per cent.
2. Set out the normalisation options open to the board with ten days available, compute what the main option would produce, and state its principal weakness.
Read the worked answer
Three options exist.
(a) Retain the raw cut-off and publish a justification. Cheapest, but the justification would have to explain why 3.4 per cent versus 7.6 per cent is rational, and the mean gap makes that hard to defend.
(b) Equal percentile (rank-based) allocation. Qualify the top 6 per cent of each shift: 0.06 × 41,200 = 2,472 candidates from Shift A and 0.06 × 27,200 = 1,632 from Shift B, total 4,104. Each shift then has its own derived cut-off (approximately 75–76 in Shift A and 82–83 in Shift B, from z = 1.555). This is transparent, computable in a day, and directly satisfies the 6 per cent band.
(c) Equipercentile or z-score equating. Transform raw scores to a common scale (for example, set both shifts to mean 60, SD 12) and apply one cut-off on the equated scale. Statistically smoother, but harder to explain to candidates and to a court in ten days.
Option (b) is the practical choice. Its weakness is the assumption it embeds: that the two shift populations are of equal average ability. If centres were assigned by subject or region and Shift B genuinely drew stronger candidates, equalising pass rates penalises them. That assumption must be checked against centre/subject composition and stated openly.
3. What should the board record in its resolution so that the decision survives a 'demonstrably rational' test? Draft the operative points.
Read the worked answer
The board should not simply announce a changed cut-off; it must create the record that shows the method was chosen for reasons, not for outcomes. Operative points:
- Findings of fact - the two shift sizes, means, SDs, the fixed cut-off, and the resulting 3.4 per cent / 7.6 per cent split, with the z-values, annexed as an appendix.
- Diagnosis - the disparity is attributable to a between-shift mean difference of 6.5 marks with comparable SDs, and shift allocation was by examination centre and not by candidate choice (state whether this is verified).
- Method chosen and why - equal percentile allocation, with reasons for rejecting raw cut-off and equating, and the derived shift-wise cut-offs recorded numerically.
- Effect - total qualifiers 4,104, within the 6 per cent band; the change applies prospectively to this result only.
- Limits - the assumption of comparable shift populations is stated, together with the check performed; candidates may represent against the derived cut-offs within a stated window.
- Future practice - normalisation method to be specified in the ordinance before the next examination, so that it is pre-committed rather than post hoc.
Point 6 is what converts a one-off rescue into a rational system; without it the same dispute recurs next year with weaker legal cover.