The Churn Model That Leaks a Protected Class
A telecom company deploys a churn-prediction model that scores every subscriber monthly. The model uses 240 features, including ZIP code, device type, and average monthly data usage. An internal audit finds that the model's precision is 0.81 overall but 0.62 for customers whose billing language is Spanish, a group that makes up 14% of the base. Digging further, the team discovers that ZIP code alone explains 38% of the model's variance and is strongly correlated with national origin in several metro areas. Removing ZIP code drops overall AUC from 0.88 to 0.79. The retention team uses the scores to decide who receives a $25 discount offer; customers not scored as high-risk get no offer. Legal has not yet weighed in, and the launch is in three weeks.
1. Identify the fairness harm(s) present and explain why removing ZIP code is not a complete remedy.
Read the worked answer
Two harms coexist. Allocative harm: the $25 retention offer is a benefit distributed unevenly - the Spanish-language group has lower precision, meaning many genuinely at-risk customers in that group are missed and get no offer. Representational/proxy harm: ZIP code is a well-documented proxy for national origin and race, so including it effectively imports a protected characteristic into the model even though the field itself is faceless.
Removing ZIP code is incomplete because:
- Redlining by proxy persists. Device type, data usage patterns, and billing language can carry the same correlation. A feature-importance rerun after removal is required, not assumed.
- The label itself may be biased. If 'churn' is measured differently for the Spanish-language group (e.g., cancellations recorded later, or involuntary churn from payment failures), the target encodes the disparity.
- AUC is the wrong yardstick for the decision. The decision is a thresholded offer, so the relevant question is who falls above the cutoff in each group, not rank-ordering quality overall.
- Dropping a feature can hurt the group it protects. If ZIP code carried real signal about coverage gaps, removing it may degrade service in exactly those areas. The remedy is a re-audit of subgroup outcomes, not a single ablation.
2. The team proposes to keep ZIP code but add a fairness constraint that equalizes offer rates across language groups. Evaluate this proposal and state what you would require before launch.
Read the worked answer
Equalizing offer rates is demographic parity, and it is the wrong constraint here for two reasons. First, the legitimate driver of offers is churn risk, and if risk genuinely differs across groups, forcing equal rates misallocates discounts - it may over-offer to low-risk customers and under-offer to high-risk ones. Second, parity on the offer does not fix the precision gap; it hides it behind a quota.
A better framing is equalized odds or calibration within group: among customers who will actually churn, the offer rate should be comparable across groups, and a given score should mean the same probability of churn regardless of language. That directly targets the 0.62 vs. 0.81 precision gap.
Before launch I would require:
- A subgroup error analysis (false negatives and false positives) at the actual operating threshold, not just AUC.
- A proxy audit: retrain without ZIP code and without the top correlated features, and compare subgroup precision.
- A documented business justification for each feature with a disparate-impact rationale.
- Legal review of the $25 offer as a consumer benefit under fair-lending-adjacent and anti-discrimination rules.
- A monitoring plan with a named owner and a rollback trigger if subgroup precision diverges by more than an agreed margin.