AU cheat sheet

An item reaches the live bank only after four recorded sign-offs: authoring, independent second-expert review, fairness/sensitivity screen, and answer-key verification.

Stem focus rule: the question must be answerable from the stem alone before options are read; if the stem only makes sense after reading the options, rewrite the stem.

Do not use NOT or EXCEPT unless the construct requires it; when used, capitalize the negative word and place it before the stem's main verb.

Options must be grammatically parallel and similar in length - a correct answer that is longer or more heavily qualified than its distractors leaks the key.

Distractor screen: each incorrect option should attract some candidates and be chosen disproportionately by low scorers; a distractor chosen by almost no one, or more often by high scorers than low scorers, is revised or replaced.

Absolute qualifiers (always, never, all, none) in options are usually incorrect and test testwiseness rather than content; replace them with qualified phrasing.

Exposure rule: rotate or retire an item once its exposure rate within a testing window crosses the security threshold; exposure is tracked alongside difficulty because shared items lose discriminating power.

Retire an item when any one of the following holds: exposure above threshold, discrimination near zero or negative, content outdated by practice or legislation, or a confirmed security breach.

Blueprint check: the percentage of items per content area on an assembled form must match the weighted content outline within the stated tolerance before publication.

Standard error of measurement: SEM = SD(observed) x sqrt(1 - reliability). A 95% confidence band is approximately observed score +/- 1.96 x SEM.

Cronbach's alpha = [k/(k-1)] x [1 - (sum of item variances / total score variance)]; alpha rises with item count and with average inter-item correlation.

KR-20 applies only to dichotomously scored items and is a special case of alpha; alpha generalizes to polytomous (partial-credit) items.

Item difficulty p equals the proportion answering correctly; for a single item, p near 0.50 maximizes score variance and potential discrimination.

Item discrimination D = p(high-scoring group) - p(low-scoring group); common screening thresholds accept D of about 0.20 or higher and flag D below 0.10 or negative.

Reliability describes consistency of scores; validity describes whether scores support the intended interpretation. High reliability is necessary but not sufficient for validity.

Equating places scores from different forms on a common scale so that a fixed cut score carries the same meaning across administrations.

Criterion-referenced cut scores are set by structured expert judgment (Angoff, Bookmark) and validated against external performance evidence, not by a fixed pass percentage.

Differential item functioning flags an item when equal-ability candidates from different groups show different correct-response probabilities; a flag triggers content review, not automatic deletion.