Blueprint Audit When Recall Items Crowd Out Application
You are auditing the draft question paper for the SSC Arabic third-language examination, total 40 marks, duration two hours. The draft's grammar section carries 20 marks: 12 marks are one-line definitions such as 'Define mubtada', 'Define fa'il' and 'What is a plural?', 4 marks are filling correct case endings into isolated words, and 4 marks are applying a rule in sentences. The prose section carries 12 marks, of which 8 are translation of single isolated words; composition is 4 marks and comprehension is 4 marks. Internal item analysis from the previous year shows that definition items had a facility of 0.92 and did not discriminate between strong and weak candidates, while the sentence-application items had a facility of 0.48 and discriminated well. The board's blueprint expects grammar 14, comprehension 10, composition 8 and translation 8, with at least half of the grammar marks at application level.
1. Identify the two most serious mismatches between the draft and the blueprint, using the item statistics as evidence.
Read the worked answer
First, grammar is over-weighted at 20 marks against a blueprint figure of 14, and the excess is almost entirely recall: 12 of the 20 marks are definitions, and only 4 are sentence application, so the paper falls well short of the requirement that at least half the grammar marks test application. The facility of 0.92 with no discrimination confirms that these items add little information about candidates. Second, comprehension and composition are under-weighted: 4 and 4 against blueprint figures of 10 and 8, while translation is heavily weighted but reduced to isolated words. So the paper is long on naming terms and short on using the language. The most serious single mismatch is the recall-heavy grammar block, because it affects 12 marks and distorts the whole profile.
2. Propose a revised 40-mark distribution that satisfies the blueprint, and show the arithmetic.
Read the worked answer
Grammar 14: 6 marks applying rules in sentences, 4 marks transformation (singular to plural, masculine to feminine), 2 marks case endings in context, 2 marks terminology used to explain an example rather than defined in isolation. Application is therefore 12 of 14, comfortably above half. Comprehension 10: 6 marks literal questions on an unseen passage, 4 marks inference and context vocabulary. Composition 8: task completion and relevance 3, grammar 3, vocabulary and expression 2. Translation 8: 5 marks sentences Arabic to Marathi, 3 marks short idiomatic phrases in context. The total is 14 + 10 + 8 + 8 = 40, and the two-hour duration is preserved because definition items are replaced one-for-one by short application items.
3. Justify why this shift improves validity without unfairly raising difficulty.
Read the worked answer
Validity improves because the paper now measures what the syllabus asks candidates to do with Arabic rather than whether they can recite labels. The statistics support the shift: recall items at 0.92 tell the examiner almost nothing, while application items at 0.48 separate candidates usefully. Difficulty need not rise unfairly because the new application items can be modelled on the textbook exercises, can carry partial credit, and can be trialled on a small sample before the paper is finalised. The total marks and time remain unchanged, and weaker candidates gain because they can earn marks for correct use and partial answers instead of losing everything on definitions they memorised imperfectly.