AU flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Content validity evidence

Evidence that test items represent the full domain defined by the blueprint, gathered through expert judgment, task analysis, or alignment studies rather than through correlation with an external criterion.

Negatively worded stems

Stems containing NOT or EXCEPT increase reading load and mis-keying risk; if unavoidable, capitalize the negative word and place it near the start of the stem.

Absolute option qualifiers

Options using always, never, all, or none are rarely correct and let testwise candidates eliminate them; rewrite them as qualified statements unless the content genuinely requires an absolute.

Plausible distractors

Incorrect options should reflect common misconceptions or computational slips so that they attract candidates who lack the knowledge; implausible fillers reduce discrimination and inflate scores through guessing.

Answer key verification

An independent second expert solves each item without seeing the proposed key; disagreement is adjudicated before the item enters the bank, catching clerical and content errors.

Fairness sensitivity review

A structured review that screens item content and language for stereotypes, unnecessary references to protected characteristics, and cultural assumptions unrelated to the construct being measured.

Construct-irrelevant variance

Score variance caused by factors the test does not intend to measure, such as excessive reading difficulty, unfamiliar contexts, or poor formatting; it lowers validity and should be removed at item review.

Item exposure rate

The proportion of candidates in a testing window who saw a given item; high exposure enables sharing and memorization, so frequently used items are retired or rotated out of the active bank.

Item bank calibration

Statistical estimation of item parameters such as difficulty and discrimination from candidate response data, used to select and assemble forms with known operating characteristics.

Test blueprint weighting

The percentage of items allocated to each content domain, set to match the importance of the domain in the job or curriculum; forms are checked against the blueprint before publication.

Two-reviewer rule

Each item is authored by one expert and independently reviewed by at least one other qualified expert before use, reducing idiosyncratic errors and single-source bias.

Item retirement criteria

Items are withdrawn when exposure exceeds a set threshold, statistics drift outside tolerance, content becomes outdated, or a security breach is confirmed.

Cognitive level tagging

Each item is labeled by the mental process it requires, such as recall, application, or analysis, so the form matches the blueprint's intended mix of higher- and lower-order items.

Stem focus rule

The question should be answerable from the stem alone before options are read; a stem that only makes sense after reading all the options is ambiguous and should be rewritten.

Homogeneous option length

Correct answers noticeably longer or more detailed than distractors give unintended cues; options should be similar in length, grammar, and structure.

Content drift review

Scheduled revalidation that checks whether items still reflect current practice, guidelines, or legislation; outdated but statistically sound items are removed on content grounds alone.

Test-retest reliability

Correlation between scores from the same candidates on two administrations separated by an interval; appropriate for stable traits and inflated by practice or memory effects over short intervals.

Cronbach's alpha

An internal-consistency estimate computed from the average covariance among items; it rises with item count and inter-item correlation and is a lower bound on reliability for a single administration.

Standard error of measurement

SEM equals the standard deviation of observed scores multiplied by the square root of one minus reliability; it estimates the spread of scores a candidate would obtain on repeated testing.

Item difficulty p-value

The proportion of candidates answering an item correctly; values near 0.5 maximize score variance, while very high or very low values provide little discrimination among candidates.

Item discrimination index

The difference in proportion correct between high- and low-scoring groups, or the item-total point-biserial correlation; low or negative values flag items for review.

KR-20 coefficient

An internal-consistency formula for dichotomously scored items; it is a special case of Cronbach's alpha and requires item-level right/wrong data rather than total scores alone.

Differential item functioning

A statistical condition in which candidates of equal ability but different group membership have different probabilities of answering correctly; flagged items undergo expert review for bias.

Angoff standard setting

Judges estimate the probability that a borderline candidate answers each item correctly; the summed probabilities form the recommended cut score, refined through discussion and repeated rating rounds.

Bookmark standard setting

Panelists place a bookmark at the point in an ordered item booklet where a borderline candidate's mastery ends; the cut score is derived from the difficulties of items at that point.

Equating

A statistical procedure that places scores from different test forms on a common scale so that the same cut score has equivalent meaning across administrations.

Scaled score

A transformed score on a defined reporting scale, derived from the raw score through equating and conversion so that scores from different forms are comparable.

Criterion-referenced cut score

A passing standard defined by the level of knowledge or skill required rather than by a fixed percentage of candidates; it is set by expert judgment and validated against performance evidence.

Distractor analysis

Examination of the proportion of candidates choosing each incorrect option; a distractor chosen by almost no one, or chosen more by high scorers than low scorers, is revised.

Reliability coefficient range

Reliability coefficients run from 0 to 1 and describe consistency, not accuracy of content; a value of 0.90 is commonly required for high-stakes decisions about individuals.

Content outline refresh cycle

Periodic job or practice analysis that surveys practitioners to confirm the tasks and knowledge areas the examination should cover; results drive revision of the test blueprint.

Form assembly constraints

Rules enforced when building a test form, including blueprint percentages, item difficulty distribution, enemy item separation, and non-overlap of content across consecutive item positions.