Predictive Welfare Scoring: Need, Privacy, and Fairness
City B has 420,000 residents, 31% aged 65 or older. The welfare department uses a predictive model to score 12,000 residents aged 75+ for risk of isolation. Inputs include utility payment records, hospital visits, and family-registry data. The top 3% are flagged for home visits by four social workers. An audit finds that 28% of flagged residents already receive services, while 35% of residents later identified as high-risk were not flagged; overall accuracy is 72%. The vendor says the model is 'objective.' A residents' group asks whether people should be scored without consent. The city has budget for either 1,500 visits a year or a smaller, opt-in system with 800 visits but better follow-up. Two council members worry that flagging will stigmatize poor households.
1. Identify the main ethical conflict and evaluate the model using at least three criteria: beneficence, privacy/autonomy, and fairness.
Read the worked answer
Main conflict. The city wants to protect isolated elderly residents, but it proposes to score them without consent using sensitive records. Beneficence pulls toward early outreach; autonomy and privacy pull toward consent and limits on data use.
Beneficence. The model may find people who would otherwise be missed. But 35% of high-risk residents are not flagged, so it cannot be the only gate. Visits must still rely on human judgement.
Privacy/autonomy. Utility and family-registry data were not collected for welfare scoring. Residents may not know they are scored, and stigma may deter contact. Opt-in respects autonomy but may miss the most isolated, who are least likely to opt in.
Fairness. The model may proxy poverty or family structure and burden poor households. A 28% overlap with existing services suggests wasted visits; false negatives raise equity concerns.
Evaluation. The model can be a triage aid, not a decision-maker. Use it to prioritize outreach, not to deny service. Add notice, consent for sensitive data, and a human review. The opt-in system is fairer but smaller; combine it with community referrals to reach non-opt-ins.
2. The vendor calls the model 'objective.' Critically assess this claim.
Read the worked answer
Calling the model 'objective' confuses statistical output with neutrality. The model's inputs are chosen by people: utility payment records, hospital visits, and family-registry data. Each has bias. Utility records may miss residents who pay in cash or live in institutions; hospital visits reflect past access, not need; family-registry data may penalize people with estranged families. The label 'high-risk' is also a human definition.
Accuracy of 72% is an aggregate. It hides false positives and false negatives, which have different costs: a false negative may leave someone without help; a false positive may stigmatize or waste a visit. The audit already shows 35% of high-risk residents missed. The vendor's claim also ignores how social workers use the score; if they trust it too much, automation bias can worsen outcomes. Objectivity would require transparency, validation across neighborhoods, and a way to contest a score. The model is consistent, but not objective in the moral sense.
3. Recommend a redesign that improves both equity and effectiveness under the budget constraint.
Read the worked answer
Redesign. Keep the model as a screening aid but separate scoring from service entitlement. Send a plain-language notice to all residents aged 75+, explaining what data are used and how to opt out. Require consent for family-registry and utility data; use publicly available or self-reported data otherwise. Have social workers review the top tier plus a random sample of non-flagged residents to catch false negatives.
Budget choice. The 1,500-visit model reaches more people but with weaker follow-up. The 800-visit opt-in system respects autonomy but may miss the isolated. A hybrid is better: 800 opt-in visits plus 400 outreach visits generated by community referrals and a simple postal check, leaving budget for follow-up. Target the 35% missed group by adding a second criterion, such as no recent clinic visit.
Equity. Do not use the score to rank need for scarce services without appeal. Publish audit results, including false-negative rates by neighborhood. This improves effectiveness and fairness without pretending the model is neutral.