HR Assistant Retrieval Over Investigation Notes
PeopleOps at a 400-person professional services firm has built an internal HR assistant using a third-party large language model API and retrieval over 12,000 SharePoint documents. The corpus includes global HR policy, compensation bands, performance-review guidance, and a folder of employee-relations investigation notes that was never meant to be searchable. The assistant uses a single service account with broad read access. The vendor’s current tier retains prompts and outputs for 30 days for “service improvement,” processes data in the United States, and offers no opt-out. The firm has employees in Germany, France, and the United States. An employee recently asked about a colleague’s medical accommodation and received a partial but revealing answer. Legal has not completed a DPIA, and the vendor contract lacks a data processing agreement. The CHRO wants to launch to all staff next month because it cut routine HR ticket volume by 38% in the pilot.
1. How should the firm classify and tier this system, and what does that classification drive?
Read the worked answer
Classify it as high internal risk regardless of whether a specific regulation labels it high-risk. The system processes employee data, including special-category health information and confidential investigation records, and it can disclose one employee’s data to another. That combination triggers GDPR obligations, confidentiality duties, and works-council or employee-representation issues in Germany and France. If the assistant were later used to support employment decisions, monitoring, or performance evaluation, it would fall squarely into the higher-risk employment category under the EU AI Act; even as a policy Q&A tool, the data sensitivity and access-control failure make the current design unacceptable.
Classification drives concrete gates: a DPIA before any expansion, a lawful basis and purpose limitation, data minimisation and retention rules, access control at the document and user level, vendor due diligence, and incident handling. It also determines who must approve go-live-legal, privacy, security, and employee representatives-not the CHRO alone. The 38% ticket reduction is a benefit, not a justification for skipping controls. The recent accommodation disclosure is a reportable incident to assess, not a bug to patch quietly.
2. What controls must be in place before any broader launch?
Read the worked answer
First, remove or quarantine the investigation-notes folder and any other collection not intended for retrieval. Re-index only approved documents. Replace the single broad service account with per-user authentication and document-level access trimming so retrieval reflects the asking employee’s entitlements; then test with adversarial queries, including attempts to retrieve colleagues’ accommodation, salary, or investigation data.
Second, complete a DPIA and establish a lawful basis, retention schedule, and data-subject request process. Minimise what is sent to the model: redact or exclude special-category data, and restrict prompts and logs. Third, fix the vendor position: a data processing agreement, documented subprocessors, no training on firm data, shorter or zero retention, and a data-residency option acceptable for EU employee data. If the current tier cannot meet that, do not launch on it.
Add operational controls: audit logging of queries and retrieved documents, a human escalation path for HR matters, user notice explaining the assistant’s limits, and an incident response playbook. Run a limited pilot with a defined group and a stop button. The accommodation disclosure should be investigated, contained, and assessed for notification obligations before scaling.
3. How should the firm handle the vendor relationship and exit risk?
Read the worked answer
Treat the vendor as a processor and negotiate from the actual data flows, not the marketing tier. Require a DPA with purpose limitation, confidentiality, breach notification timelines, audit or assurance rights, subprocessor approval, and deletion or return of data on exit. Ask specifically whether prompts and retrieved documents are used for model training, how long they persist, where they are stored, and whether EU-only processing is available. Get answers in the contract, not in a sales call.
Build an exit and fallback plan before launch: the ability to export the document index and query logs, a tested process to switch providers, and an internal-only mode for sensitive queries. Avoid embedding the vendor’s API so deeply that turning it off halts HR operations. Define service levels and remedies for data-handling failures, and require notification if the vendor changes retention or subprocessors.
The firm should also decide which queries are out of scope for any external model-investigation notes and accommodation data should probably never be sent. If the vendor cannot meet the baseline, delay the launch and use the pilot’s ticket-reduction evidence to justify a compliant procurement rather than accepting unacceptable risk for speed.