Agent Assists Order Exceptions but Must Not Act Alone
An operations team wants an AI agent to handle order-exception triage. The agent reads exception queues, decides whether an order should be cancelled, re-shipped, or escalated, and can call Boomi processes to execute those actions. Roughly 3,000 exceptions occur daily; about 6% are genuinely ambiguous and need a human. Leadership wants the agent to auto-resolve the clear cases to cut a 14-hour manual backlog. Compliance requires that every cancellation be attributable to a named human or an approved automated rule, and that no agent action be irreversible without a recorded rationale. The team has no evaluation set and no logging of agent decisions today.
1. Design the control model for this agent. Where does it act autonomously, where does it stop, and what must exist before go-live?
Read the worked answer
Treat autonomy as a graduated permission, not a binary. The agent may act autonomously only on exception types where the decision rule is deterministic and the action is reversible - for example, re-shipping a replacement, which can be reversed by cancelling the replacement. Cancellations are irreversible from the customer's perspective and carry a compliance attribution requirement, so they should require either a human approval or execution under a pre-approved, documented rule that names the rule rather than the agent as the actor.
Before go-live, three things must exist. First, an evaluation set: a labeled sample of historical exceptions with the correct disposition, used to measure the agent's agreement rate and, critically, its error distribution - a false cancellation is far costlier than a false escalation, so the threshold should be set on the costly error, not overall accuracy. Second, decision logging: every agent decision records inputs consulted, the disposition chosen, confidence, and rationale, written before the action executes so a failed action still leaves a record. Third, an escalation path that is genuinely reachable - the 6% ambiguous cases must route to a human queue with the agent's reasoning attached, not just a flag.
Start the agent in a recommend-only mode against live traffic to gather agreement data, then enable autonomous action per exception type as each clears its threshold.
2. Six weeks in, the agent's escalation rate has climbed from 6% to 22%. How do you investigate, and what failure modes do you rule out?
Read the worked answer
A rising escalation rate means either the input distribution changed or the agent's confidence calibration drifted; distinguish them before changing anything.
Compare the exception mix now versus baseline. If a new exception type or a new upstream data shape appeared, the agent is correctly escalating something it was never evaluated on, and the fix is coverage, not tuning. Check input completeness next: a missing field the agent relies on - a carrier status, a customer flag - will push otherwise-clear cases into ambiguity. This is common when an upstream system changes and is easy to miss because the agent still returns an answer.
Then examine the escalations themselves. If the agent escalates cases that humans resolve identically every time, the confidence threshold is mis-set or the prompt/rule set drifted. If escalations are genuinely hard, the rate may be honest and the earlier 6% was overconfident - which the evaluation set should reveal.
Rule out two failure modes explicitly: silent tool failures, where a Boomi process call errors and the agent escalates rather than acting, and feedback contamination, where human resolutions of escalated cases are not fed back, so the agent never learns the cases it punted. Check action-call error rates and whether resolved escalations re-enter the evaluation data.