HR Agent: Generative Answers Across 4,000 Employees
A 12-person HR shared-services team handles 1,800 tickets a quarter; roughly 60% repeat ten questions about leave balances, payslip location, and policy documents. They built a Copilot Studio agent with generative answers pointed at the HR SharePoint site and the public careers site, and piloted it on a Teams channel open to the whole company. In the pilot the agent answered "how much leave do I have left?" by quoting the leave policy, and it surfaced a draft restructuring FAQ from a documents library to a user outside HR. The team wants to launch to all 4,000 employees in three weeks.
1. Explain why the leave-balance question failed and what must change before launch.
Read the worked answer
Generative answers retrieve and summarise published content; the leave policy is content, but an individual's balance is a per-user data lookup, so the agent answered the only question it could. No amount of extra knowledge sources or prompt tuning fixes this.
It needs an authored topic with a defined trigger phrase, an authenticated user context, and a Power Automate flow or connector call that queries the HR system for that specific employee's balance and returns a structured answer. The same pattern applies to payslip location if the answer is a personal link.
This also sets expectations for launch scope: generative answers can carry the ten policy-style questions, but the personalised ones must be deliberately built and tested. Launching in three weeks is feasible only if the team builds the two personalised topics first and lets generated answers cover the rest.
2. The restructuring FAQ leak: what went wrong and what is the correct control?
Read the worked answer
The agent was pointed at a site or library whose contents included material not intended for general staff. Generative answers will use whatever is in the configured sources, so the control is source selection, not the model.
Correct it by removing the broad site as a knowledge source and replacing it with curated, published sources - specific pages or a dedicated library that HR controls - and by verifying with test questions that draft documents and restricted libraries are not retrievable. Do not rely on the assumption that the agent inherits every SharePoint permission for every user across every channel.
The judgement lesson is that a knowledge source is a publication decision. Anyone who can add a document to that library can effectively publish to 4,000 people through the agent.
3. What must be true before opening the agent to all 4,000 employees?
Read the worked answer
Three things gate the launch. First, authentication: on Teams, users are identified, which the personalised topics require; if a web or public channel is added later, those topics must degrade to a hand-off rather than answering with someone else's data.
Second, an escalation path: a clear route to a human when the agent is unsure or when the question touches something sensitive like a restructure, plus a way to record that the conversation was escalated.
Third, measurement and a kill switch: analytics on unresolved questions and topic performance so the team can see whether the 60% repeat volume actually falls, and the ability to disable a knowledge source without redeploying the agent.
Because the reputation risk sits in one bad answer to 4,000 people, launch to a department first, review the transcripts, then widen.