A Test That Measured the Wrong Thing
Fadi teaches Grade 7 mathematics at a boys' school in Amman. His 34 students sat a 40-item unit test he wrote by copying items from the textbook's end-of-chapter exercise bank. The blueprint was 28 one-step computational items, 8 word problems and 4 conceptual items. Results: mean 47%, with 22 of 34 students below 50%. Only 3 students solved the two-step word problem correctly, but 30 solved the decimal division item. Quizzes during the unit averaged 78% because they were announced and students could use their notes. Two students who scored above 90% on the quizzes scored 55% and 48% on the test. The school requires a mark out of 100 for the first-term report in two weeks, and the head of department wants the syllabus finished. Fadi proposes to add 10 points to every mark 'so the parents don't complain'.
1. Interpret the evidence. Distinguish what the data do and do not support, and evaluate the proposal to add 10 points to every mark.
Read the worked answer
What the data support.
- The class can execute taught procedures under familiar conditions: 30 of 34 on decimal division.
- Transfer is weak: 3 of 34 on a two-step problem.
- The 78% quiz average and the 47% test mean are not a contradiction; they measure different things, because the quizzes were announced, open-note and near-identical to practised items.
- The drop for the two high quiz scorers (90+ to 55 and 48) is a signal about those students' dependence on cues, not evidence that they cheated.
What the data do not support. They do not show that the class 'failed to learn' or that the test was too hard in a general sense - the computational items were largely correct. They also cannot rank students reliably, because a 40-item test with 28 near-identical procedural items gives a narrow, skewed score distribution.
The +10 proposal. Reject it. It is unrelated to learning, it is applied equally to students who need different things, it conceals a real gap in multi-step reasoning, and it will be visible to parents comparing results with other classes. It also creates a precedent: next term the same problem returns with no information gained. If the school's reporting rules require a mark, report the achieved mark and attach a short skills profile so the number is interpretable.
2. Redesign the assessment plan for the remaining two weeks. Give a table of specifications for the replacement test and show how the eight available periods are used.
Read the worked answer
Periods available: 2 weeks × 4 = 8. Allocation: 2 periods re-teaching the concept behind the procedure with concrete and pictorial models; 1 period on word-problem schema (identify knowns, unknown, operation); 2 periods on two- and three-step problems with worked examples and faded guidance; 1 period on error analysis of their own test scripts; 1 period for a formative quiz plus targeted re-teaching; 1 period buffer.
Specifications for the replacement test (30 items, 45 minutes).
| Content | Recall/procedure | Application (multi-step) | Reasoning/explanation | Total |
|---|---|---|---|---|
| Decimals and fractions | 5 | 4 | 1 | 10 |
| Ratio and proportion | 4 | 4 | 1 | 9 |
| Expressions and equations | 4 | 4 | 1 | 9 |
| Representation (table, graph) | 0 | 2 | 0 | 2 |
| Total | 13 | 14 | 3 | 30 |
Design rules. No item copied verbatim from the textbook; at least half the application items set in unfamiliar contexts; one item per objective marked against a short rubric; marks allocated per step so partial reasoning is visible.
Checkpoints before the test: a 10-minute quiz in each of the two weeks, unannounced but low stakes, marked and returned within 48 hours, used to decide the re-teaching in the final week.
3. What should Fadi do for the 22 students below 50%, given that the report is due in two weeks and the syllabus is unfinished?
Read the worked answer
Separate the two problems. The reporting deadline and the syllabus are administrative; the reasoning gap is instructional. Solving the first by inflating marks leaves the second untouched and returns next term.
Instructional actions. Group the 22 by error type rather than by score: those who can compute but cannot set up a problem (likely the majority, given the drill results), those with place-value or decimal errors, and the few with both. Run a 10-minute targeted start to each of the next six lessons on the dominant error type, with worked examples and immediate checking. Use paired work with mixed attainment only after the weaker students have had explicit modelling, otherwise they copy rather than reason. Give the two high-quiz/low-test students a specific task on unsupported problem solving, since their difficulty is cue dependence.
Reporting. Enter the achieved marks. Attach a one-paragraph profile per student naming one secure skill and one target skill, and note on the class summary that the assessment blueprint was revised after item analysis, so the mark is not comparable to the previous quiz average. Inform the head of department in writing that the syllabus will be completed one week later and why. Inform parents at the reporting point with the profile, and invite the 22 to a short after-school session.
What not to do: do not curve, do not drop the lowest scores, and do not re-test the same items and count the higher score as the term mark.