Exposure Creep on a Retired Form
A nursing specialty exam reuses a 40-item equating block across forms. After a form is retired, its items remain in the live pool as unscored pretest items for two cycles. A candidate who tested in January posted 60 recalled items to a public forum; 22 of them are in the current equating block. The program's security policy requires retiring any item with confirmed exposure, but the equating block is the only anchor linking the current form to the scale. Removing 22 of 40 anchors would leave 18. The program also has a candidate agreement prohibiting disclosure but no active monitoring. The next administration is in three weeks, and 900 candidates are registered.
1. Quantify the threat and decide whether to remove the 22 items. What is the minimum anchor requirement and how would you test it?
Read the worked answer
Threat assessment: 22 of 40 anchors (55%) are publicly recalled. Recalled items are not perfect reproductions, but the exposure is real and asymmetric - candidates who study the forum gain an advantage concentrated on the very items used to place scores on the scale. That corrupts equating, which is worse than corrupting a few scored items.
Anchor sufficiency: there is no universal minimum, but common practice treats fewer than 20-25% of a form as an anchor as fragile, and 18 of 160 scored items (11%) is thin. The decision should be evidence-based, not count-based:
- Recompute the equating with 18 anchors and compare the resulting scale conversion to the 40-anchor version. If the difference in cut-score-equivalent raw score is within a pre-set tolerance (e.g., ≤1 raw-score point), 18 anchors are adequate for this cycle.
- Check anchor representativeness - do the surviving 18 span the blueprint and difficulty range? A difficulty-skewed 18 is worse than a representative 18.
- If tolerance is exceeded, do not simply use 18. Options: substitute vetted items from the prior form's anchor set, or delay the equating decision and use a documented fallback (e.g., hold the scale via mean/sigma on a reduced anchor with a sensitivity flag).
Recommendation: remove the 22 confirmed-exposure items immediately. They cannot be trusted as anchors. Then run the 18-anchor equating and apply the tolerance test.
2. What immediate and structural security actions would you take, and what would you tell the 900 registered candidates?
Read the worked answer
Immediate (within days):
- Remove the 22 items from scoring and from the pretest pool; quarantine the remaining 18 anchors pending the tolerance test.
- Issue a takedown request to the forum and preserve evidence (timestamps, screenshots, account identifiers) for any enforcement action.
- Audit whether the 22 items appeared on other live forms; if so, expand the quarantine.
- Brief the 900 candidates only to the extent necessary: confirm the exam will proceed, that security procedures are being followed, and that no candidate action is required. Do not disclose which items or the anchor structure - that compounds exposure.
Structural (before the next cycle):
- End the practice of keeping retired-form items in the live pretest pool; pretest items should be new or from forms never publicly administered.
- Diversify anchors across multiple forms so no single exposure event can remove half the anchor set.
- Add active monitoring (forum and social-media sweeps) with a defined response protocol.
- Strengthen the candidate agreement with a specific consequence clause and enforce it in at least one visible case.
Document the entire sequence for the security incident log and the next program review.