The Hiring Screener With a Clean Audit and a Dirty Pipeline
A mid-sized logistics firm uses a third-party resume-screening tool to rank applicants for warehouse supervisor roles. The vendor's audit shows no disparate impact by race or gender at the 0.8 four-fifths threshold. The firm's own review finds that the top 10% of ranked candidates are 71% male and 82% from three metro areas, while the applicant pool is 48% female and spread across 14 metros. The tool was trained on 12 years of the firm's prior hires, and until 2019 the firm explicitly recruited supervisors through a referral program dominated by current male employees. The vendor says its model is 'facially neutral' because it uses only job-relevant features like years of experience and shift flexibility. A recruiter notes that women in the pool disproportionately list caregiving gaps, which the model penalizes.
1. Explain how a model can pass a four-fifths audit yet still produce a skewed shortlist.
Read the worked answer
The four-fifths rule is a coarse screen at one decision point, and it can pass while harm is concentrated elsewhere:
- The audit may test the wrong stage. Passing at the 'recommended' cutoff says nothing about the composition of the top 10%. If the firm only interviews the top decile, the operative disparity is at that rank, not at the pass/fail line.
- Small or aggregated groups hide skew. A pooled race/gender audit can pass while metro-level or intersectional subgroups fail. The 71%/82% figures are exactly the kind of concentration a pooled audit misses.
- The training label is contaminated. The model learned from 12 years of hires, including the referral era. It is reproducing the outcome of a biased pipeline, not the requirements of the job. 'Facially neutral features' do not help when the label encodes past exclusion.
- A proxy is doing the work. Caregiving gaps correlate with gender. Penalizing employment gaps is a neutral-sounding rule with a disparate effect, and the recruiter has already observed it.
Passing four-fifths is necessary at best, not sufficient.
2. The vendor offers to add a 'gender balance' post-processing step that re-ranks to hit 50/50 in the top 10%. Evaluate this and propose an alternative.
Read the worked answer
Re-ranking to a fixed 50/50 is demographic parity bolted on at the end, and it has three problems. It treats gender as the only axis of concern while the metro skew (82% from three cities) goes untouched. It can promote candidates who are not job-qualified to hit the quota, which invites a legitimate challenge and erodes trust in the tool. And it leaves the contaminated label and the caregiving-gap penalty in place - the model still believes gaps are disqualifying; it is just overridden at the last step.
A better alternative:
- Fix the target. Define supervisor success from actual post-hire performance data (retention, safety incidents, throughput), not from who was hired during the referral era. Retrain on that.
- Remove or justify the gap penalty. Test whether employment gaps predict performance at all. If they do not, drop the feature. If they do, document the job-relatedness and monitor the gender effect.
- Audit at the operating point. Measure selection rates at the top-10% cutoff by gender, race, and metro, including intersectional cells.
- Fix the pipeline, not just the ranker. Broaden sourcing beyond the three metros and the referral network; the shortlist skew is partly an input problem.
- Keep a human in the loop with structured, job-related review criteria for anyone near the cutoff.
Post-processing can be a stopgap, but it must not substitute for repairing the label and the sourcing.