DPIA for an AI Screening Tool with a Tight Deadline
A recruitment firm plans to deploy an AI CV-screening tool that ranks candidates and auto-rejects the bottom 40%. It will process 50,000 applications a year, including special category data inferred from CVs (e.g., health mentions, ethnicity inferred from names or photos). The vendor claims the model is 'bias-tested' but will not share the training data or feature weights. The tool will be hosted by a sub-processor in a country with no adequacy decision and no standard contractual clauses yet signed. The client wants to go live in 4 weeks. The DPO has been asked to sign off a one-page risk assessment instead of a full DPIA because 'the vendor has a privacy policy'.
1. Determine whether a DPIA is required and justify your conclusion.
Read the worked answer
A DPIA is required. The processing meets multiple high-risk triggers:
- Systematic and extensive evaluation of individuals using automated processing, including profiling, with significant effects (auto-rejection).
- Special category data processing, including inferred health and ethnicity data.
- Innovative technology (AI/ML) with opacity.
- Large scale: 50,000 applications per year.
- Vulnerable context: job applicants are in an imbalance of power.
- Possibly automated decision-making with legal/significant effects, triggering Article 22 considerations.
A one-page risk assessment is not a substitute. The DPIA must describe the processing, assess necessity and proportionality, identify risks to rights and freedoms, and set out mitigations. The vendor's privacy policy is irrelevant to the controller's DPIA obligation. The DPO should refuse sign-off and escalate; going live without a DPIA where required risks regulatory action and is a governance failure.
2. Identify the principal risks and propose mitigations that could allow a compliant deployment, including the transfer and sub-processor issues.
Read the worked answer
Principal risks:
- Bias and discrimination from auto-rejection and inferred ethnicity/health; opaque model prevents validation.
- Lack of explainability for rejected candidates; Article 22 and fairness concerns.
- Unlawful special category processing without an Article 9 condition.
- International transfer without a transfer mechanism.
- Sub-processor chain without contractual controls.
- Accuracy of inferences.
Mitigations:
- Require the vendor to provide model documentation, bias testing methodology, and feature weights under NDA; commission an independent audit.
- Remove auto-rejection: use ranking as decision support with human review of all rejections; provide meaningful information to candidates.
- Establish an Article 9 condition (e.g., explicit consent where appropriate) or configure the tool to avoid inferring special category data; strip photos and names before screening.
- Complete a transfer risk assessment and sign SCCs plus a supplementary measures assessment before go-live; identify the sub-processor and obtain authorisation.
- Update the privacy notice and run a candidate-facing transparency exercise.
- Set a pilot with monitoring, a bias dashboard, and a kill switch.
If these cannot be completed in 4 weeks, recommend delaying go-live; the DPIA should record residual risk and the DPO's advice.