The Resume Ranker With No Bias Audit
TalentBridge, a US-headquartered HR platform, sells an AI resume-ranking module to employers. It is marketed in the EU, New York City, Illinois, and Brazil. The model is trained on 12 million historical applications from US clients; rankings are delivered as a 1-100 'fit score' with a short natural-language rationale. One EU client uses it to screen 40,000 applicants for warehouse roles; a NYC employer uses it for 900 senior finance roles. TalentBridge has no bias audit, no candidate notice, and its contract says clients are 'solely responsible for compliance.' A journalist has asked for the model's training data. Leadership asks whether to pause sales.
1. Characterize TalentBridge's and its clients' roles under the EU AI Act and NYC Local Law 144. Which characterization is contestable?
Read the worked answer
EU AI Act: Employment and worker-management AI - including recruitment and candidate selection - is high-risk under Annex III. TalentBridge is likely a provider because it places the system on the market under its own name. Its clients are deployers. If a client substantially modifies the model or puts its own name on it, that client can become a provider. The high-risk classification triggers provider obligations such as a risk-management system, technical documentation, logging, human oversight design, accuracy/robustness/cybersecurity, and conformity assessment; deployers have obligations around use, monitoring, and human oversight.
NYC Local Law 144: The tool is likely an automated employment decision tool because it substantially assists or replaces discretionary hiring decisions. The employer must have an independent bias audit within the prior year, publish a summary of results, and provide candidate notice. A vendor supplying the tool has a practical role in furnishing data and documentation, but the statute places the audit and notice duties on the employer.
Contestable points: whether the 'fit score' is actually used to substantially assist decision-making; whether the natural-language rationale changes the analysis; whether the EU client's warehouse screening is 'substantially modifying' the system; and whether the vendor or client is the provider for the EU deployment. These are fact-dependent and should be resolved before sales continue.
2. Can TalentBridge rely on the contract clause shifting compliance to clients? Explain with reasoning about contractual versus regulatory responsibility.
Read the worked answer
No, not as a shield against regulators. A contract can allocate indemnities, warranties, and commercial risk between private parties. It cannot displace statutory duties that attach to a provider or deployer by law. If TalentBridge is a provider under the EU AI Act, it owes provider obligations regardless of a clause saying clients are 'solely responsible.' A regulator enforces the law, not the contract.
Practical limits of the clause:
- It may be unfair or unenforceable in some jurisdictions, especially where it purports to shift all regulatory responsibility to a party that lacks the information to comply.
- It does not solve the NYC audit problem: the employer bears the audit duty, but the employer cannot conduct a credible audit without vendor data and model documentation. A clause that withholds that data makes compliance impossible and invites client claims.
- It creates misalignment: the vendor has the model knowledge; the client has the use context. Compliance requires both.
Better approach: replace the clause with a responsibility matrix. TalentBridge supplies training-data documentation, model cards, bias-audit inputs, and change notices; clients own use-specific notices, human oversight, and deployment monitoring. Indemnities should track actual control, not a blanket disclaimer.
3. Propose a 90-day plan addressing the journalist inquiry and the sales pause decision.
Read the worked answer
Days 1-7 - Stabilize and preserve. Issue a litigation/regulatory hold; do not delete training data, model versions, or sales records. Route the journalist inquiry to legal and communications; give a factual holding statement, not a defense of the product. Pause new EU and NYC sales until the gap assessment is complete; honor existing contracts but flag risk to clients.
Days 8-30 - Assess. Map each deployment by jurisdiction, role, and use. Determine whether the tool is high-risk under the EU AI Act and an AEDT under NYC LL144. Review training-data provenance, consent/licensing, and representativeness. Interview the EU and NYC clients about actual decision use. Identify whether any client has substantially modified the tool.
Days 31-60 - Remediate. Commission an independent bias audit with a documented methodology; publish the summary where required. Draft candidate notices and client-facing transparency materials. Amend contracts to a responsibility matrix and add data-sharing for audits. Implement model documentation, change control, and logging.
Days 61-90 - Decide and communicate. Resume sales only in jurisdictions where the compliance package is complete, or restrict to a documented pilot. Brief the board on residual risk and the journalist response. Establish a quarterly review. The key judgment: a sales pause is cheaper than a regulator finding that the company sold a high-risk employment tool without the required documentation.