The Hiring Screener Under Regulatory Scrutiny
A company uses a machine learning model to rank job applicants for interviews. The model was trained on five years of historical hiring decisions. An internal audit finds that the interview rate for one demographic group is 40% lower than for another after controlling for qualifications in the data. The model's top features include graduation year and a zip code. The company operates in a jurisdiction that requires explainability for automated employment decisions. Leadership asks whether to remove the zip code feature and relaunch, or pause the system.
1. Evaluate the proposal to simply remove the zip code feature and relaunch.
Read the worked answer
Removing one feature is not a sufficient remedy and can create a false sense of compliance.
First, zip code is likely a proxy for protected characteristics. Removing it does not remove the information: graduation year, school names, and other features can encode the same signal, so the disparity may persist or shift to other features. Proxy variables are the core difficulty in fairness work.
Second, the disparity was measured in outcomes after controlling for qualifications. That is the symptom regulators care about, and it must be re-measured after any change, not assumed fixed.
Third, the model was trained on historical hiring decisions. If past decisions were biased, the model learned those patterns, and dropping a column does not undo that. Retraining on the same labels reproduces the same bias.
Finally, the jurisdiction requires explainability. Removing a feature does not produce explanations for affected applicants, nor does it address the documentation and audit trail the regulation expects.
A defensible path: pause or restrict the system, investigate feature provenance and label quality, retrain with fairness constraints or reweighted labels, re-audit outcomes by group, and build the explainability and documentation the jurisdiction requires before relaunch.
2. What governance steps should accompany any relaunch, and how should the company decide whether to pause now?
Read the worked answer
Pause now if the system is actively making or influencing decisions while a known, measured disparity exists and the jurisdiction requires explainability the company cannot yet provide. Continuing to deploy converts an audit finding into ongoing exposure.
Governance steps for relaunch:
Assign accountability: a named owner for the model, a review board that includes legal and a fairness reviewer, and documented decision authority.
Document the system: intended use, training data provenance, known limitations, the fairness metric chosen and why, and the audit results. This supports the explainability requirement.
Define acceptance criteria before relaunch: a measured disparity threshold by group, explainability output for each affected applicant, and a monitoring plan with periodic re-audits.
Add human review: a human makes the final interview decision, with the model as an input, and applicants can request an explanation.
Set monitoring and rollback: track selection rates by group over time, alert on drift, and define who can halt the system.
Decide to pause based on severity and reversibility: a measured 40% gap in a regulated decision is high severity, so pause while remediating rather than relaunch with a cosmetic feature change.