Fairness Metric Conflict in Hiring Screening
A technology company uses a machine-learning model to screen 38,000 applicants for engineering roles and invites the top 15% to interview. The applicant pool is 22% female and 9% underrepresented minority. The model's selection rate is 17% for female applicants and 29% for male applicants; demographic parity difference is 0.12. Equal opportunity difference, measured at the same label threshold, is 0.03. Two strong features are 'years of experience' and 'university prestige,' both correlated with gender. Business leaders want to increase female representation; legal counsel warns that hiring quotas and direct use of protected attributes are unlawful. The model owner proposes lowering the score threshold for female applicants by 0.08 to close the gap. The company operates in the United States and must document adverse impact.
1. Interpret the fairness metrics and the proposed threshold change.
Read the worked answer
The two metrics answer different questions. Demographic parity difference of 0.12 means selection rates differ by 12 percentage points (17% versus 29%), which signals adverse impact but does not by itself prove unfairness. Equal opportunity difference of 0.03 means the model's true-positive rates are close across groups at the same label threshold, so the model ranks qualified applicants similarly.
Lowering the female threshold by 0.08 would directly use gender in selection, which legal counsel warns is unlawful and would create a quota-like process. It also treats gender as the cause rather than addressing job-relatedness or proxy features. The better question is whether the model's features are job-related and whether the adverse impact ratio is acceptable. Investigate proxy features and job requirements before adjusting thresholds.
2. Recommend mitigation and validation steps.
Read the worked answer
Mitigation should focus on construct validity and process, not protected-class thresholds. Remove or transform 'university prestige' if it is not job-related and acts as a proxy; validate 'years of experience' against performance data. Consider structured interviews, work samples, and skills-based assessments.
Run adverse impact analysis by gender and race, calculate selection-rate ratios, and test alternative model specifications. If a gap remains, use a legally reviewed, job-related adjustment such as expanding the interview pool or targeted outreach, not group-specific score thresholds. Validate on a holdout and monitor selection rates, performance, and retention. Document the job-relatedness evidence and the tradeoff between demographic parity and equal opportunity.
3. What documentation and governance should accompany the model?
Read the worked answer
Document the model's purpose, features, label definition, validation, fairness metrics, and known limitations. Record the adverse impact analysis and the legal review of any mitigation.
Governance should include a model owner, approver, monitoring cadence, complaint handling, and an audit trail. Do not use protected attributes as inputs; keep them only for fairness testing under controlled access. Set review triggers: selection-rate ratio below 0.8, material change in applicant pool, or feature drift. Report to the board or diversity committee with both parity and equal-opportunity metrics, and state that no quota is being applied. If the model cannot be made job-related and non-discriminatory, fall back to structured human review.