Adventure Works: An HR Agent Facing Shutdown After Eight Weeks
Adventure Works rolled out an HR policy agent to 900 employees eight weeks ago. Adoption is 12%, 34% of questions return 'I don't know', and the only feedback mechanism is a thumbs-up button nobody reads. There is no named product owner. Last week an employee pasted an external email into the chat that contained an instruction to ignore prior rules, and the agent summarised the injected text as if it were policy. The CHRO wants to shut the agent down; the IT lead wants to expand it to finance and legal. No evaluation set exists, and the licence renewal is in 30 days.
1. Is 12% adoption with 34% unresolved a quality problem, an awareness problem, or both? What would you measure in the next two weeks?
Read the worked answer
You cannot currently answer the question, and that is the finding. Twelve percent adoption could mean employees cannot find the agent, or tried it and left. Thirty-four percent unresolved could mean thin content, poor retrieval, or questions the HR function has never answered in writing.
Two weeks of measurement: cluster the last 500 conversations by intent and rank unresolved ones by frequency. Check whether each top intent has an authoritative source document at all - many 'I don't know' answers are honest, because the policy genuinely does not exist. Check reachability: is the agent pinned in Teams, named in onboarding, and linked from the intranet? Compare adoption in teams that received a launch briefing against those that did not.
Build a 150-question golden set from real HR tickets with known answers, and measure retrieval recall separately from answer correctness. That distinguishes missing content from missing retrieval from bad generation, and each has a different fix. Add an explicit 'was this helpful' with a reason code, and read a sample weekly. Without these three instruments, shutdown and expansion are both guesses.
2. How do you handle the prompt injection incident and set the responsible AI controls that expansion would require?
Read the worked answer
Treat the incident as a scope failure, not a model failure. An HR policy agent should answer from indexed HR knowledge, never from arbitrary pasted content treated as policy. Blast radius was limited because the agent takes no actions, but the same weakness in finance or legal would be serious.
Controls: restrict answers to the indexed corpus and use pasted text only as a search query, never as an instruction; apply an instruction hierarchy so system rules outrank user content; add output filtering and a refusal path when content conflicts with the corpus; log the full conversation for the incident review. Publish an acceptable-use note telling employees not to paste external or confidential content.
Add adversarial cases to the regression set now - embedded instructions, requests for another employee's data, attempts to extract the system prompt - and re-run them on every release. Name an accountable owner for the agent; without one, none of these controls have a person behind them. Expansion to finance and legal is off the table until the injection class of tests passes and a data-access review shows the agent respects existing entitlement boundaries.
3. Renewal is in 30 days. What is your recommendation and on what conditions?
Read the worked answer
Recommend a conditional renewal at current scope, explicitly not an expansion, with a 30-day evidence window and a named owner as a precondition. The CHRO's instinct to shut down and the IT lead's instinct to expand are both premature; the data does not yet support either.
Conditions to renew: an owner appointed from HR with IT support; the 150-question golden set live; an explicit feedback mechanism with reason codes; ingestion of the missing policy documents identified from the top unresolved intents; and the prompt-injection test suite passing.
Decision thresholds at day 30: containment above 60%, unresolved rate under 15%, and a helpfulness score above 4 out of 5 on a sampled basis. If thresholds are missed but trending, renew one more quarter at reduced scope. If unresolved remains above 25% with no content gap explanation, decommission and keep the knowledge base improvement, which has standalone value.
Expansion to finance and legal is a separate decision requiring its own discovery, data classification, action-safety design, and evaluation set. Bundling it into this renewal would repeat the original mistake: launching before the measurement existed.