The Fairness Audit That Found a Proxy
A bank uses a model to shortlist small-business loan applications for manual review. The model was trained on five years of historical approvals. An internal audit finds that approval rates for applicants in majority-Black ZIP codes are 14 percentage points lower than for applicants in majority-white ZIP codes with similar revenue and years in business. The model does not use race or ZIP code directly. It does use 'average balance volatility over 24 months', 'number of overdrafts', and 'business age'. The bank's compliance team says the model is 'race-blind, therefore fair'. The model also drives a marketing pre-approval list.
1. Assess the compliance team's 'race-blind, therefore fair' argument.
Read the worked answer
The argument is wrong on both statistical and regulatory grounds.
Statistically: not using race does not prevent disparate impact. Features like balance volatility and overdraft counts can act as proxies when their distributions differ by group for reasons unrelated to creditworthiness - e.g., cash-flow timing in businesses with thinner buffers. A 14-point gap at similar revenue and tenure is exactly the pattern a proxy produces. 'Race-blind' describes inputs, not outcomes.
Regulatorily: fair-lending frameworks (e.g., disparate impact doctrine under ECOA in the US, and similar principles elsewhere) assess outcomes, not intent. A model that produces a large adverse gap can be a fair-lending problem even if race never appears as a field. The audit finding is the evidence a regulator would ask for.
What to do: confirm the gap is not explained by legitimate credit factors via a controlled analysis; test whether the proxy features add predictive value beyond legitimate ones; consider less-discriminatory alternatives; document the analysis.
2. The bank wants to keep the model and 'just add a race field so we can measure fairness.' What is right and wrong with that, and what would you actually do?
Read the worked answer
Right: you need group-level outcome data to measure fairness. You cannot audit what you don't measure.
Wrong: adding race as a model input is both legally risky in many jurisdictions and technically misguided - it doesn't fix proxy effects and may create explicit disparate treatment. Adding race as a monitoring dimension is different from using it as a feature; these must not be conflated.
What to do:
- Separate monitoring from modeling. Use a protected, access-controlled attribute for fairness measurement only, never as a feature. Confirm with legal counsel what is permissible in the relevant jurisdiction.
- Explain the gap. Decompose the 14-point difference across features. Is it driven by the proxy features, by missing legitimate variables, or by label bias in historical approvals?
- Mitigate. Options include removing or transforming proxy features, reweighting training data, or applying group-specific thresholds - each with tradeoffs to be documented.
- Govern the marketing use. The pre-approval list amplifies any disparity; it needs its own fairness review and likely a hold until the model is remediated.
- Document the analysis and decisions for the regulator.