Loan Approval Model Bias and Fairness
A bank has deployed a machine learning model to approve or deny personal loans. The model was trained on 5 years of historical loan data. An internal audit found that the model's approval rate for female applicants is 62%, while for male applicants it is 78%. The bank operates in a jurisdiction where discrimination based on sex in credit is illegal. The model's overall accuracy is 85%, and it uses features including income, credit score, employment length, and zip code. The data science team argues that the disparity is justified because the model is simply reflecting historical lending patterns and that removing gender and zip code would reduce accuracy. The compliance team is concerned about legal and reputational risk.
1. Identify the type of bias present and explain how it might have arisen from the data and features used.
Read the worked answer
The disparity in approval rates between female and male applicants indicates disparate impact (a form of bias). Even if gender is not used as a feature, the model can learn proxy discrimination from correlated features. Zip code can serve as a proxy for race or gender due to residential segregation. Historical lending patterns may have favored male applicants, and the model learns these patterns. Additionally, if the training data underrepresents female applicants or has label bias (e.g., women were denied loans historically due to biased human decisions), the model perpetuates that bias. The use of zip code, which correlates with protected characteristics, can introduce indirect discrimination. The team's argument that removing gender and zip code reduces accuracy is not a valid justification under the law; fairness may require accepting some accuracy loss or using mitigation techniques.
2. Recommend concrete steps to mitigate the bias and ensure compliance, considering technical and organizational measures.
Read the worked answer
- Data audit: Analyze training data for historical bias and underrepresentation. Consider reweighting or resampling to balance outcomes. 2. Feature engineering: Remove or transform proxy features like zip code. Use fairness-aware feature selection. 3. Model mitigation: Apply pre-processing (e.g., disparate impact remover), in-processing (adversarial debiasing, fairness constraints), or post-processing (equalized odds) techniques. 4. Evaluation: Use fairness metrics (demographic parity, equal opportunity) alongside accuracy. Set thresholds for acceptable disparity. 5. Governance: Establish a fairness review board, document decisions, and monitor outcomes continuously. 6. Legal review: Ensure compliance with anti-discrimination laws; consult legal experts. 7. Transparency: Provide adverse action reasons to applicants. 8. Human oversight: Allow manual review for borderline cases. The goal is to reduce disparity while maintaining model utility, possibly through retraining with fairness constraints.