Psychometric Analysis and Scoring
Psychometric Analysis and Scoring
Psychometrics ensures exam scores are reliable and valid. Key concepts include reliability (consistency, e.g., Cronbach’s alpha, KR-20), validity (evidence supporting score interpretation), and item statistics: difficulty (p-value) and discrimination (point-biserial or D-index). A p-value near 1.0 indicates an easy item; near 0.0, a hard item. Discrimination indices above 0.30 are generally acceptable; negative values signal a flawed item. Distractor analysis examines how many candidates chose each option; a distractor chosen by no one is nonfunctional. Scoring can be dichotomous (right/wrong) or polytomous (partial credit). Equating places scores from different forms on a common scale (e.g., via common items or IRT). Standard error of measurement (SEM) quantifies score precision. A common trap is confusing reliability with validity: a test can be reliable but not valid. Another is misinterpreting p-values as percentages of correct responses (they are proportions). IRT models (e.g., Rasch, 2PL, 3PL) provide item difficulty, discrimination, and guessing parameters. Differential item functioning (DIF) analysis detects items that behave differently for subgroups after matching on ability. Worked example: If an item has p = 0.90 and point-biserial = 0.10, it is too easy and poorly discriminating; consider revision. Rule of thumb: flag items with p < 0.20 or > 0.90, and discrimination < 0.20 for review.