The DPIA That Was Signed Off in a Day
A financial-services firm plans to deploy an AI model that scores loan applicants and auto-declines the bottom 15%. The model is trained on five years of historical application data including postcode, age band, and prior repayment history. The vendor supplied a 'bias-tested' certificate. The project manager completed a 6-page DPIA in one day, marked residual risk 'Low', and obtained sign-off from a delivery director. No DPO opinion is recorded. No data subjects were consulted. The model will go live in three weeks. A privacy engineer reviewing the file notes: (a) the DPIA never identifies the lawful basis for the profiling; (b) there is no analysis of the postcode feature as a proxy for ethnicity; (c) no explanation of the logic is documented; (d) the 'Low' rating is asserted without a scoring method. The firm operates in the UK and Ireland.
1. Is a DPIA required here, and is this DPIA adequate? Justify both.
Read the worked answer
Required: yes. This is a textbook trigger case - systematic and extensive evaluation of individuals based on automated processing (profiling) producing legal or similarly significant effects (auto-decline of credit), plus large-scale processing of financial data and innovative technology. Any one of these would likely trigger a DPIA; together they make it mandatory. The fact that a DPIA exists does not settle the question - its quality does.
Adequate: no. The file fails on multiple counts:
- No lawful basis. A DPIA must describe the processing and its necessity and proportionality; you cannot assess proportionality without naming the basis. For automated credit decisions the firm must also confirm the Article 22 position - is this solely automated decision-making with significant effects? If so, safeguards (human intervention, contest, express views) must be designed in.
- Unaddressed proxy discrimination. Postcode as a feature is a well-known proxy risk. The DPIA must analyse it, not rely on a vendor certificate; the certificate is evidence, not a substitute for the controller's own assessment.
- No explanation of logic. Meaningful human review and the right to an explanation both depend on documented model logic. 'Black box' is not an answer.
- Risk rating without method. 'Low' with no scoring framework is an assertion, not an assessment.
- No DPO advice recorded. The DPO must be consulted; their advice and any disagreement should be documented.
- No data subject views sought. Where appropriate, the DPIA should seek the views of data subjects or their representatives.
Conclusion: the DPIA is a shell. It must be redone before go-live, and the three-week timeline should slip if it cannot be.
2. The firm wants to go live in three weeks. Set out a defensible path to launch, and say what would make launch indefensible.
Read the worked answer
Defensible path (three weeks is tight but not impossible if scope is controlled):
- Week 1 - rebuild the assessment. Name the lawful basis; confirm Article 22 applicability and design safeguards (human review of declines, contest route, explanation). Add a proxy-discrimination analysis of postcode and age band, with testing done by the firm or an independent party, not just a vendor certificate.
- Week 1–2 - mitigation design. Consider removing or transforming the postcode feature; document the decision either way. Define explainability requirements and how a declined applicant receives a meaningful explanation. Set a real risk-scoring method (likelihood × severity) and re-rate.
- Week 2 - consultation. Obtain and record DPO advice; seek data subject or representative views where appropriate; if residual risk remains high after mitigation, consult the supervisory authority before proceeding.
- Week 2–3 - sign-off and monitoring. Senior sign-off with the DPO opinion attached; a monitoring plan for model drift and disparate impact; a review date.
Indefensible: launching on the existing DPIA with 'Low' asserted, no lawful basis, no proxy analysis, and no DPO opinion - even if a director signs it. Sign-off by a delivery director does not cure a deficient assessment, and 'the vendor certified it' is not a defence. If residual high risk cannot be mitigated in time, the correct answer is to delay, not to launch.