The 50-Question Final at Riverside Community Center
Marcus teaches a 10-week, 20-hour 'Money Basics' course at a community center for 38 adult learners, most referred by a workforce agency. He must report completion numbers to the county funder, which pays per graduate. His predecessor used a 50-item multiple-choice final with a 70% pass mark, but the item bank was written at roughly a 12th-grade reading level. Last cohort, 19 of 34 learners scored below 70%, and 11 of those 19 had missed three or more sessions. Marcus wants to keep the same final because it already exists, and a colleague suggests simply lowering the pass mark to 60%. The current group includes two learners who are deaf and rely on an ASL interpreter, several who are still in ESOL classes, and one learner with a mild cognitive disability who has attended every session and participates actively. The funder's contract requires 'demonstrated competency' but defines nothing further.
1. Diagnose the problems with the current assessment plan. What decisions must be made before any test is rewritten?
Read the worked answer
Three separate problems are being bundled into one test score.
- Validity. A 50-item multiple-choice final written at 12th-grade reading level measures reading as much as money knowledge. If the course objectives are things like 'build a monthly budget' or 'read a pay stub,' a recall test does not sample the domain at all.
- Confounding with attendance. Eleven of the 19 failures missed three or more sessions. Those learners may never have been taught the content; failing them measures exposure, not ability. Lowering the cut score to 60% merely relabels that problem.
- Purpose confusion. Marcus is using one instrument for three jobs - certifying an individual, reporting a completion count to a payer, and evaluating his own course. These require different evidence. Paying per graduate adds a conflict of interest that makes a soft cut score tempting and indefensible.
Decisions that must come before redesign: What must a graduate be able to do, stated observably? Is the program certifying competency or recording participation? What will the funder actually accept as evidence? And what accommodations are required - the two deaf learners and the ESOL learners are entitled to an assessment that measures the construct rather than their English or their hearing.
Lowering the pass mark fails all three tests: it fixes neither validity nor the attendance confound, and it converts a credential into a number that means nothing to anyone.
2. Propose a redesigned assessment plan. Tie each element to the specific failure it addresses.
Read the worked answer
Start with a one-page blueprint. Three to five competencies, each with an observable performance. Example: given a pay stub and two bills, produce a balanced monthly budget and identify one trade-off.
Assessment menu, matched to competency:
- Performance tasks scored with a checklist: build a budget, read a pay stub, compare two loan offers using APR, find the fee on a statement. This is the core evidence and it addresses the validity failure directly.
- A short plain-language item set (8–12 items at 6th–8th grade reading level, reviewed for cultural assumptions) replacing the 50-item bank. Keep the good items; discard the rest.
- Oral or conferenced demonstration for learners who can show the skill verbally, scored on the same rubric. This is an accommodation, not a lower standard - it fixes the construct-irrelevant barrier for the ESOL and deaf learners.
- Portfolio option: a completed budget brought back the following week.
Process changes:
- Two attempts with targeted re-teaching between them; mastery models vary time, not the standard.
- Rubric with three levels, and a second rater on a sample to check consistency.
- Attendance reported separately ('completed 7 of 10 sessions') so it stops silently determining pass/fail.
- Pre/post administration for program-level evidence, which is what course evaluation actually needs.
Cost: roughly six hours of item rewriting plus ten minutes per learner for performance scoring. Compare that with the cost of a credential nobody trusts.
3. How should Marcus report to a funder that pays per graduate and asks for 'demonstrated competency' without defining it?
Read the worked answer
Report two metrics, never one. Completion equals attended at least seven of ten sessions. Competency equals met the rubric on the performance tasks. They answer different questions and the funder is entitled to both.
Write the definition into the contract before the cohort starts. If 'graduate' is undefined at intake, it will be defined retrospectively by whoever is under financial pressure in June.
Be explicit about denominators and cut scores. A sentence such as: of 38 enrolled, 29 completed the course; 24 of those 29 demonstrated competency on all four performance tasks within two attempts; five required a third attempt is defensible, auditable, and still shows a respectable rate. Vague percentages invite the funder to assume the worst.
Guard against the payment incentive. Three protections: (1) fix the definition of graduate in advance; (2) never let the person whose payment depends on the count be the sole scorer - a second rater on a sample is cheap insurance; (3) report attempts, not just final results, so the standard's integrity is visible.
If the funder pushes back, explain that a credential everyone receives carries no information for the funder's own outcome reporting, and offer the rubric as a shared definition of success. Walk them through the performance tasks; program officers usually accept a defensible standard once they can see it.
Document accommodation decisions in aggregate (how many, what type) without naming learners, so the record shows the standard was applied fairly rather than uniformly.