Fairness Audit Before a Hiring Screening Rollout
A company plans to deploy a resume-screening model that ranks applicants for interview. On a historical dataset, the model selects 30% of male applicants and 18% of female applicants for the top rank band. Overall selection rate is 24%. The legal team cites the four-fifths rule, which flags selection ratios below 0.8. The model's defenders note that the training labels were human hiring decisions, so the model reproduces past practice, and that a feature correlated with gender, years of continuous experience, drives much of the gap because career breaks are more common among female applicants. The company wants to keep predictive validity while reducing adverse impact, and must decide before a pilot with 5,000 applicants.
1. Compute the relevant disparity measure and assess the situation under the four-fifths rule, noting its limits.
Read the worked answer
The selection ratio is the protected group's rate divided by the reference group's: 18% / 30% = 0.6, below the 0.8 threshold, so the four-fifths rule would flag adverse impact. The overall 24% rate is not the comparison that matters. Limits: the rule is a screening heuristic, not a legal verdict; small samples make ratios unstable, and here the pilot of 5,000 will produce more reliable estimates than the historical set if group sizes are known. The rule says nothing about whether the gap is justified. The defenders' point that labels encode past human decisions is important: the model may be reproducing bias rather than predicting merit, so high validity against biased labels is not a defense. The experience feature is a plausible proxy, but correlation with gender does not by itself prove the feature is improper; it requires analysis.
2. Propose concrete options to reduce adverse impact while preserving useful prediction, and evaluate the trade-offs.
Read the worked answer
Options: (1) Remove or transform the continuous-experience feature, for example measuring relevant skills or recency-weighted experience that does not penalize career breaks; this targets the proxy but may cost validity if experience truly predicts performance. (2) Reweight training labels or resample to correct the historical human bias; this changes what the model learns and can improve parity, but the labels themselves may be noisy. (3) Post-processing: choose group-specific thresholds to equalize selection rates; effective for the ratio but may lower overall validity and raises legal and ethical questions about explicit group treatment. (4) Redefine the target using job-performance outcomes rather than prior hiring decisions, which addresses the root cause but requires outcome data and time. (5) Use the model only to assist human reviewers with structured rubrics and audit decisions. The best near-term mix is usually (1) plus (4) as a longer project, with (3) considered only with legal guidance.
3. What would you require before the 5,000-applicant pilot proceeds, and how would you measure success?
Read the worked answer
Before the pilot: document the target definition and confirm labels reflect job-relevant outcomes where possible; run subgroup error analysis, not just selection rates; test whether the experience feature's removal changes both disparity and validity; set pre-registered thresholds for acceptable selection ratio (for example at least 0.8) and minimum validity; obtain legal review of any group-specific thresholds; and plan human oversight so no applicant is rejected solely by the model. During the pilot, log decisions, track selection ratios by group with confidence intervals, monitor for proxy features, and collect outcome data to validate the target. Success means the selection ratio meets the agreed threshold without unacceptable validity loss, no subgroup shows materially worse false-negative rates, and human reviewers confirm rankings are job-relevant. If the ratio cannot be met without gutting validity, pause and revisit the target rather than shipping.