Ballari lecturer's 40-mark mid-term: scaling, z-scores and two suspicious items
A lecturer at a government pre-university college in Ballari has finished a 40-mark mid-term test in Education and must now use it for part of the 20-mark internal assessment. Section A (n = 40) has a mean of 26.5 and an SD of 6.2; Section B (n = 38) has a mean of 24.8 and an SD of 9.4. Ananya (Section A) scored 34; Bharath (Section B) scored 21. Seven students in Section A scored below the 40% pass mark of 16.
She also ran an item analysis, taking the upper and lower 27% groups (11 students each). Item 7: 9 of the upper group and 3 of the lower group answered correctly. Item 12: 2 of the upper group and 1 of the lower group answered correctly; the topic had been taught in the last two periods before the test, and this was the first question she had ever set on it.
Because seven students failed, the lecturer proposes adding 5 marks to every script so that nobody fails, arguing that this will also "raise the class average". The principal asks her to justify the decision in writing before the IA marks are uploaded.
1. Convert Ananya's and Bharath's raw scores into z-scores and T-scores. What does each score mean within the section, and why can the two sections' means not be compared directly?
Read the worked answer
Working
- z = (X − M)/SD; T = 50 + 10z.
- Ananya: (34 − 26.5) = 7.5; 7.5 ÷ 6.2 = +1.21; T = 50 + 12.1 = 62.
- Bharath: (21 − 24.8) = −3.8; −3.8 ÷ 9.4 = −0.40; T = 50 − 4.0 = 46.
Interpretation
Ananya is about 1.2 SD above her section mean, roughly the 88th percentile of Section A. Bharath is 0.4 SD below his section mean, roughly the 34th percentile. Raw marks (34 vs 21) suggest a gap of 13 marks; in standard units the gap is about 1.6 SD. The raw-mark comparison is misleading.
Why the means cannot be compared directly
The SDs differ sharply (6.2 vs 9.4). Section B is far more heterogeneous - it contains both very high and very low scorers - so a similar class mean conceals a different distribution. Three conditions must hold before comparing means: the same paper and scoring, comparable testing conditions, and comparable spread. Only the first is likely to hold here.
What to report
Report to students and parents: a within-section position (z, T or percentile) plus a criterion-referenced statement of which units the student has actually mastered, taken from item-level data. A single class average should never be the headline figure when SDs differ this much.
2. Evaluate Item 7 and Item 12 using the difficulty index and discrimination index. What should the lecturer do with each item?
Read the worked answer
Formulae
- Difficulty p = (R_upper + R_lower) ÷ (n_upper + n_lower)
- Discrimination D = (R_upper − R_lower) ÷ n_upper
Item 7
- p = (9 + 3) ÷ 22 = 12 ÷ 22 = 0.55
- D = (9 − 3) ÷ 11 = 6 ÷ 11 = 0.55
A moderately difficult item that separates the upper and lower groups very well. Retain it in the question bank; it is doing exactly the diagnostic work a test item should do.
Item 12
- p = (2 + 1) ÷ 22 = 3 ÷ 22 = 0.14
- D = (2 − 1) ÷ 11 = 0.09
Too hard and almost no discrimination: the upper group barely did better than the lower group, so the item is not measuring the construct for anyone. Do not use it for IA as it stands. Before blaming the students, test four competing explanations: (i) a miskey - check the answer key against the scripts; (ii) ambiguous wording; (iii) the topic was taught in the last two periods, so it tests recent exposure rather than learning; (iv) the item asks something never practised. Re-teach, then re-test with a reworded item.
Caution on the numbers
With only 11 students per group, D is unstable - one student moving groups shifts D by about 0.09. Treat both indices as flags for review, not verdicts, and never discard an item purely on a computed value.
3. Assess the proposal to add 5 marks to every script. Does it achieve the stated aims, and what defensible alternatives exist?
Read the worked answer
What adding a constant actually does
- Mean rises by 5 (26.5 → 31.5); the seven students cross 16 and "pass".
- SD stays 6.2; the rank order is completely unchanged; the distance of every student from every other is unchanged.
So the proposal scores 1 out of 2 on its own stated aims: it raises the average but changes nothing about relative performance. Worse, it breaks the criterion-referenced meaning of the test. A pass mark is a claim that the student can do specified things; adding 5 marks asserts competence that has not been demonstrated. It also erases the diagnostic signal that seven students have a real, probably localised, gap - and the inflated figures go into an auditable IA record.
Defensible alternatives, in order of preference
- Audit the paper. If Item 12-type items are defective, drop them and re-scale uniformly out of the reduced total, with the working shown.
- Reteach and re-test the weak unit; keep both sets of marks, with the second used for IA.
- Give item-level feedback so each of the seven knows which unit to repair.
- Pre-declare any moderation formula in writing, apply it to all sections, keep raw scores on file, and get the principal's approval.
- Refer persistently low scorers for remedial support or counselling - do not convert a learning problem into an arithmetic one.
A lecturer cannot unilaterally rewrite IA marks; KSEAB and departmental procedure govern this, and the justification note should say so.