Public-Sector Procurement: Certification Claims, Prohibition Risk and the DPIA Gap

No. ISO/IEC 42001 certifies that the supplier has an AI management system in place; it does not certify CaseAssist’s conformity with the EU AI Act, and “EU AI Act ready” is a marketing phrase, not a legal status. SOC 2 covers controls relevant to security, availability and confidentiality, not AI-specific risks such as bias, human oversight or fundamental rights. Due diligence should test the product, not just the management system. Ask for: the intended-purpose statement; the high-risk classification analysis; technical documentation; accuracy, robustness and bias test results; human-oversight design; logging and retention; data provenance and whether GPAI subcontractor terms flow down; model card with known limitations; change-management and incident-notification commitments; audit rights; and subprocessor list. CivicWorks should also run its own DPIA because it will fine-tune on 10 years of case notes and rank individuals. The 30-day deadline is a procurement constraint, not a legal excuse. A practical approach is a two-stage gate: stage 1 before contract signature (classification, DPIA screening, prohibition screen, security and data-flow review); stage 2 before go-live (bias testing, human-oversight procedure, staff training, monitoring plan). If the vendor will not provide enough evidence, the correct decision is no award or a pilot with strict limits, not reliance on a certificate.

Classification must start with the intended purpose and the decision the tool influences. CaseAssist ranks housing-benefit cases by fraud risk and suggests investigation priority. If it is used to evaluate eligibility for essential public assistance benefits, it is likely Annex III high-risk (access to essential services). If it only prioritises already-qualified cases for investigation, the analysis is less clear, but the tool still profiles individuals and may materially influence a detrimental decision. The prohibition screen is critical: Article 5 prohibits social scoring by public authorities where individuals are classified based on social behaviour or personal characteristics and treated detrimentally in unrelated contexts or in a disproportionate way. A fraud-risk score built from ten years of case notes, used to prioritise investigations, could fall close to that line, especially if it uses data unrelated to the benefit claim. That is a legal judgement that should be documented and, if needed, escalated. GDPR applies separately: profiling and large-scale special-category or financial data require a DPIA before processing; a legal basis and data-minimisation analysis are needed. The DPIA should cover necessity, proportionality, explainability, human review and redress. The civil-society concern is a signal to test, not a public-relations problem.

Recommended contract clauses: purpose limitation and a defined intended purpose; compliance with the AI Act and GDPR; provider cooperation with conformity assessment and authorities; audit and inspection rights, including subprocessors; incident notification within 24 hours and root-cause reporting; model-change notification and revalidation; data ownership, retention and deletion; restrictions on using CivicWorks data to train vendor or GPAI models; bias and accuracy testing evidence; human-oversight requirements; service levels for explainability and redress; termination and exit assistance; indemnities for regulatory fines caused by vendor breach; and flow-down to the GPAI subcontractor. Go/no-go: do not sign on the current evidence. Issue a conditional award subject to: a completed classification and prohibition memo, a DPIA, satisfactory bias and accuracy evidence, a named CivicWorks product owner and human-review process, and contract clauses on audit and incident notification. If the prohibition analysis concludes social scoring, stop. If the tool is high-risk, require a conformity-assessment plan and do not go live before the required CE marking and registration. The procurement team should record the decision and the residual risk accepted, so the 30-day deadline does not become the reason for an unexamined launch.