ACE-Expert study plan
ACE-Expert · Give each week a clear focus.
8 weeks · About 7 hours per week
Item and Question Development
Draft 12 original four-option single-best-answer items mapped to Domain 2 objectives
Classify each drafted item by cognitive level (recall, application, analysis) and flag imbalance
Write a one-page item-writing checklist covering stem clarity, plausible distractors, and cueing avoidance
Revise at least 6 drafted items using the checklist and log each change
Your checkpoint
A 12-item draft set with cognitive-level tags and a documented revision log
Item and Question Development & Exam Development and Governance
Convert 4 drafted items into case-based or scenario formats with self-contained context
Peer-review a colleague's or your own item set against a fairness and bias screen
Map every item to a single primary domain and record the rationale
Identify 3 governance touchpoints (item review, approval, retirement) that affect item quality
Your checkpoint
A finalized 12-item bank with domain mapping, fairness screen notes, and governance touchpoint list
Psychometric Analysis and Scoring
Compute difficulty (p-value) and discrimination (point-biserial) for a sample item response dataset
Interpret distractor frequency distributions to flag nonfunctioning options
Explain how item statistics feed form assembly and flagging rules
Summarize classical test theory concepts (true score, reliability, standard error) in your own notes
Your checkpoint
A one-page item analysis report on a sample dataset with flagged items and recommended actions
Psychometric Analysis and Scoring & Standard Setting and Cut Score Validation
Distinguish norm-referenced and criterion-referenced scoring and when each applies
Describe the modified Angoff method step by step, including panel composition and rounds
Draft a 10-step standard-setting panel agenda with time allocations
Compare at least two standard-setting methods on assumptions and evidence needs
Your checkpoint
A standard-setting panel agenda plus a one-page method comparison table
Standard Setting and Cut Score Validation
Compute a cut score from a small simulated Angoff rating set and document the aggregation rule
Outline evidence sources for cut score validation (internal, external, procedural, consequential)
Design a cut score review cycle with triggers for reconsideration
Draft a defensible rationale narrative for a hypothetical cut score decision
Your checkpoint
A cut score validation evidence matrix with at least 8 evidence sources and their collection methods
Test Security and Fairness
Inventory test security threats across pre-administration, administration, and post-administration phases
Draft a security incident response procedure with escalation steps and timelines
Identify fairness considerations in item content, accessibility, and differential performance
Write a candidate confidentiality agreement outline and a test-center irregularity log template
Your checkpoint
A security and fairness playbook covering at least 10 threats, mitigations, and response actions
Program Evaluation and Quality Improvement & Exam Development and Governance
Define 5 program-level quality metrics with formulas and target thresholds
Design a plan-do-study-act cycle for one identified program weakness
Draft a governance charter outline covering roles, decision rights, and review cadence
Map the exam lifecycle from job analysis to score reporting and identify quality checkpoints
Your checkpoint
A program evaluation dashboard mock-up with 5 metrics, data sources, and thresholds
Exam Development and Governance & Item and Question Development & Psychometric Analysis and Scoring & Standard Setting and Cut Score Validation & Test Security and Fairness & Program Evaluation and Quality Improvement
Complete a 60-question timed practice exam and score by domain
Analyze missed items by domain and root cause (knowledge gap vs. misread vs. guess)
Re-review the two lowest-scoring domains using your own notes and flashcards
Write a post-exam remediation plan with dates for any remaining weak areas
Your checkpoint
A scored practice exam report with a domain-by-domain remediation plan
Milestones
Week 2: 12 original items finalized with domain mapping and fairness screen documented
Week 4: Item analysis report complete and standard-setting panel agenda drafted
Week 6: Cut score validation matrix and security/fairness playbook both complete
Week 8: Timed practice exam scored at or above your target with a written remediation plan