Two Assessors, One Weld, and a Missing PPE Step
In an automotive welding program, 24 candidates complete a 90-minute practical assessment for a mandatory competence unit. Two assessors score independently using a 20-item rubric. On one criterion - 'prepares joint and applies PPE before striking arc' - Assessor A records 4/4 for Candidate 17 because the final weld passes visual inspection and the candidate wore a helmet and gloves; Assessor B records 2/4 because the candidate did not fasten the leather apron until after the first tack weld, a step listed in the rubric as 'before arc.' Candidate 17 otherwise scores high. The workplace supervisor, who observes the candidate daily, says the apron lapse was 'a one-off' and asks the program not to fail the learner. The assessment policy requires all critical safety criteria to be met at 3/4 or above; the rubric's wording does not define 'before striking arc.' The appeals window opens in two days. The program coordinator asks for a defensible decision.
1. Analyze whether the assessment instrument and process provide valid, reliable evidence for this candidate. Identify the specific flaws and their consequences.
Read the worked answer
The core problem is construct validity and inter-rater reliability. The rubric phrase 'before striking arc' is not operationalized, so Assessor A treats PPE as a global condition of the task while Assessor B treats it as a strict sequence. Both readings are defensible; that ambiguity, not the candidate's weld quality, produces the 2/4 versus 4/4 discrepancy. Because the criterion is classified as critical and policy requires at least 3/4, the disagreement is high-stakes rather than a minor scoring variation.
The supervisor's testimony is useful corroboration but is not sufficient assessment evidence. It was not gathered under controlled conditions and may reflect production habits rather than the stated standard. The candidate's high scores elsewhere cannot compensate for an unmet critical safety criterion; critical criteria are usually non-compensatory. The process also lacks pre-scoring calibration or moderation, so independent scoring without a shared interpretation increases variance.
Consequences: the result is vulnerable on appeal, the candidate receives inconsistent feedback, and the program cannot confidently defend either score. A defensible process would define the observable behavior, require assessor calibration, and provide a moderation step before results are released. The instrument should state what must be worn, when it must be worn, and what counts as evidence that the sequence was followed.
2. What immediate outcome should the program reach for Candidate 17, and how should it be documented and communicated before the appeals window opens?
Read the worked answer
The program should not certify the candidate yet, but it also should not simply uphold Assessor B's 2/4 on an ambiguous rubric. Certifying on the basis of the passing weld would ignore a critical safety standard; failing solely on Assessor B's reading would be procedurally unfair because the criterion was not clear.
The defensible outcome is 'not yet competent' pending a targeted reassessment after the criterion is clarified. Before the appeals window, the coordinator should convene both assessors and a moderator, rewrite the criterion in observable terms, record the candidate's account, and arrange a short conditions-controlled observation of donning PPE and preparing the joint before the arc. The candidate should receive written notice explaining the ambiguity, the safety requirement, the reassessment conditions, and appeal rights.
If the candidate repeats the omission under clear instructions, the critical criterion is not met and the not-yet-competent decision stands. If the candidate meets the sequence, the new evidence supports competence. The supervisor's request should be acknowledged politely but not used to waive the standard. Documentation should include the original scores, the rubric ambiguity, the moderation decision, the reassessment evidence, and the rationale. This protects fairness, safety, and the credibility of the qualification.
3. Recommend three program-level improvements to prevent similar disputes in future practical assessments.
Read the worked answer
First, rewrite critical criteria as observable, binary or clearly anchored behaviors. For example: 'Fastens leather apron, helmet, and gloves before the electrode or torch is energized; assessor observes the sequence directly.' Each score point should describe what the candidate does, not an inferred quality.
Second, introduce mandatory assessor calibration and moderation. Before each assessment cycle, both assessors should score the same video or live demonstration and reconcile differences. A sample of at least 10% of assessments, plus all failed critical criteria, should be moderated before release.
Third, strengthen evidence and appeal procedures. Use a standard record sheet with a checklist of PPE sequence, a brief candidate declaration, and space for assessor notes. Train assessors on non-compensatory critical criteria and on separating product quality from process safety. Publish the reassessment and appeal process in learner-facing language. These measures improve reliability without lowering the practical standard.