Northline Hardware: A Valid Test With Adverse Impact and an Unevaluated Onboarding Program
Northline Hardware operates 140 stores and hires roughly 2,400 cashiers a year; first-year turnover is 65%. Over three months, HR piloted a 25-item situational judgment test on 480 applicants. Scores correlated .38 with supervisor performance ratings at 90 days (statistically significant), and the test costs $6 per applicant to administer. Pass rates, however, differed by group: 78% of white applicants, 61% of Hispanic applicants, and 58% of Black applicants passed. No job analysis was conducted before the test was built; the items were adapted from a generic retail bank the vendor supplied. In a separate initiative, all new hires attended a two-day onboarding program. Its only evaluation is an end-of-class reaction survey averaging 4.6 out of 5, and 90-day turnover has not moved. The VP of HR wants to roll both programs out chain-wide next quarter and has asked you for a recommendation.
1. Compute the adverse impact ratios and advise whether Northline may adopt the test as it stands.
Read the worked answer
Arithmetic. Compare each group's pass rate to the highest group's rate: 61 ÷ 78 = 0.78 for Hispanic applicants; 58 ÷ 78 = 0.74 for Black applicants. Both fall below the 0.80 threshold in the Uniform Guidelines on Employee Selection Procedures, which means the test shows adverse impact and the employer bears the burden of showing the device is job-related and consistent with business necessity.
Why validity alone does not save it. A .38 correlation with supervisor ratings is respectable for a single predictor, but under Title VII disparate-impact doctrine a demonstrably valid test can still be unlawful if a plaintiff shows an equally valid alternative with less adverse impact was available. The employer also has to defend the criterion itself: supervisor ratings at 90 days are subjective and may carry their own bias, which weakens the whole validation story.
What is missing. There was no job analysis, so there is no evidence the items measure cashier competencies rather than generic retail folklore. The vendor bank was not built for this job.
Recommendation: do not roll out next quarter. Next steps: (1) conduct a job analysis and build or revise items from actual critical incidents; (2) re-validate on a larger sample with a criterion that is not purely supervisory, such as cash variance, transaction accuracy, or absenteeism; (3) examine less impactful alternatives or a composite (structured interview plus work sample) and compare both validity and impact; (4) check statistical significance of the pass-rate differences at these sample sizes before assuming the gap is stable. A .38 predictor is worth pursuing, but not in this form.
2. Design an evaluation of the onboarding program that would actually tell the VP whether it works.
Read the worked answer
The reaction survey measures only whether trainees liked the class. It cannot show that turnover should have moved, so the 4.6 out of 5 is evidence of nothing except pleasant delivery.
Use the four Kirkpatrick levels with a defensible design:
- Level 1, reaction: keep it, but add questions about perceived job relevance, not just satisfaction.
- Level 2, learning: a short knowledge check on POS procedures, loss prevention, and product basics at the end of day two.
- Level 3, behavior: a supervisor observation checklist at 30 days covering the specific behaviors the class was supposed to change. If onboarding never targeted a behavior, that is the real finding.
- Level 4, results: 90-day turnover, 90-day sales per labor hour, cash shortage per cashier, and first-30-day procedural errors.
Design matters more than the checklist. Because all new hires attend, any before/after comparison is confounded by season, store, and manager. The strongest feasible option is a stepped-wedge rollout: assign stores to two or three waves and compare trained cohorts against not-yet-trained cohorts during the same weeks, controlling for store and manager. Randomizing individual hires within a store is weaker because contamination across coworkers is likely.
Then do the cost-benefit. At 2,400 hires and 65% first-year turnover, roughly 1,560 separations occur each year. If a separation costs $2,500, that is $3.9 million. A credible ten-point improvement in 90-day turnover saves about $390,000 a year, which is the number that should decide whether the program scales, not the 4.6.
3. The VP argues that 65% turnover is unavoidable in hourly retail and that the money is better spent on faster hiring than on selection or onboarding. How would you respond?
Read the worked answer
Challenge the premise with the cost structure rather than with opinion. Sixty-five percent turnover in a job that pays roughly $15 an hour is high even for hourly retail, and the annualized cost is the largest controllable line item in the store labor budget.
Separate the turnover into avoidable and unavoidable before accepting either argument. Pull separation reasons by tenure band. If a large share of exits occurs in the first 90 days, the cause is selection or onboarding, and faster hiring simply moves people through the door faster. If exits cluster at 9–14 months, the driver is scheduling, pay, or supervision, and neither selection nor speed of hire will touch it. If exits are dominated by students and seasonal workers, part of the 65% genuinely is structural.
Then price the alternatives honestly. Faster hiring improves time-to-fill and reduces vacancy hours, which is real value; it does not reduce the replacement cost per separation. A validated selection device plus structured onboarding attacks the replacement cost, and the savings scale with every hire. These are complements, not competitors, and the staged rollout described above is cheap enough to run while hiring speed work proceeds.
Finally, note the sequencing constraint: adoption of the test is legally blocked until the job analysis and impact questions are resolved, so "spend it on hiring speed instead" is not a choice between two available options this quarter.