Trade Secrets and Due Process in a Child Welfare Risk Tool
Riverton County's child protective services agency uses a vendor's FamilyRisk algorithm to prioritize investigations. The model produces a 1-100 risk score from 87 inputs, including prior reports, Medicaid claims, and address changes. Caseworkers must document reasons for overriding a score above 70. The contract contains a trade-secret clause prohibiting disclosure of the model's logic. An audit finds that 62% of children removed from homes had scores above 80, but the false-positive rate is 45%. Families receive a letter stating that an algorithmic assessment indicates elevated risk without specific factors. A parent requests the reasons for her score and an appeal. The vendor says disclosure would reveal proprietary information. State law requires due process before temporary custody, and the county attorney warns that the current letters may be legally insufficient.
1. Analyze the transparency and explainability obligations owed to families, balancing trade secrets and due process.
Read the worked answer
The family has a strong liberty interest in the care and custody of a child, and the risk of erroneous deprivation is high: a 45% false-positive rate means the score can be wrong in a large share of cases. Due process requires notice and a meaningful opportunity to be heard, and that in turn requires an explanation the family can understand and contest. The county does not necessarily have to disclose source code or the vendor's full model logic. It must disclose the gist of the evidence: the score, the score band, the principal factors that drove it, the data sources, and how the family can challenge errors. Trade-secret protection can be accommodated through protective orders, disclosure to a neutral decision-maker or expert, and redaction of genuinely proprietary details. A bare statement that an algorithmic assessment indicates elevated risk is not meaningful notice because it gives the family nothing to rebut. Ethically, transparency is owed not only because it is legally required but because affected people are entitled to know the basis of a state action that can remove a child. The county should also disclose the tool's known error rates and its role in the decision; the human caseworker remains the decision-maker.
2. Design an appeal process that gives families a meaningful opportunity to be heard without disclosing the vendor's proprietary model.
Read the worked answer
A workable process should include:
- Timely notice: plain-language notice within 24 hours of a score being used, stating the score band, the principal factors, the data sources, and the family's right to appeal.
- Access to the case record: the family and counsel can see the reports and data relied on, with protective redactions where necessary.
- Explanation: the agency provides a local explanation identifying the top factors, not the full proprietary model. If the vendor refuses, the county should require explanation as a contractual condition.
- Neutral hearing: a hearing before an independent officer within 72 hours, with the right to counsel, to present evidence, and to cross-examine the agency's evidence.
- Protected disclosure: the vendor can disclose model details to the hearing officer or a neutral expert under a protective order, without public release.
- Written decision: the officer explains the evidence relied on and the reasons, and the caseworker must independently justify any continued removal.
- No score-only decisions: the score cannot be the sole basis for removal.
This design protects trade secrets while giving the family a meaningful chance to correct errors. It also creates a record for judicial review.
3. Who is accountable for harms from the tool, and what governance changes should the county adopt?
Read the worked answer
The county remains accountable. It is the government actor making the custody decision, and it cannot delegate constitutional obligations to a vendor. The vendor is accountable for contractual warranties, accuracy claims, and cooperation with audits, but the county owns the decision and the consequences. Governance changes should include human review before any removal, a documented override process, independent validation and adverse-event reporting, and audits of false-positive rates by race, disability, and income. Procurement should require explainability, audit rights, performance reporting, and termination rights. The county should also create a model risk committee, train caseworkers on the tool's limits, publish aggregate performance data, and involve families and community advocates in oversight. Contracts should not allow trade-secret claims to defeat due process. Finally, the county should track harms after removal and provide a route for correction and remedy. Accountability means being able to answer who decided, on what evidence, with what oversight, and what happened afterward.