Support triage: supervisor with typed handoffs vs. free-form agent bus
Helios Support runs a four-agent assistant for a consumer-electronics retailer: a router, a billing specialist, a returns specialist, and a policy-QA agent. Volume is 6,000 tickets per day, and the p95 time-to-first-token target is 2.5 s. After three weeks in production, 18% of tickets are routed to the wrong specialist and bounce back to the router, and each bounce costs one extra full LLM turn. Average inference cost per ticket has climbed to $0.42. Duplicate tool calls - the same order lookup executed twice by two different agents - appear in 27% of traces. Engineers propose replacing the current free-form message bus, where every agent sees the entire transcript, with a supervisor that emits typed handoffs plus a shared scratchpad of verified facts. Product wants to keep the current topology because "the agents collaborate better". Audit review also found no recorded reason for any routing choice.
1. Is the 18% misroute rate a collaboration problem or a routing problem? Recommend a topology and justify it against Product's objection.
Read the worked answer
The 27% duplicate-call figure is the real evidence. When every agent reads a full free-form transcript, each one re-derives state instead of consuming a decision, so work is repeated and the router never receives a signal that its choice was wrong. The 18% is a classification failure amplified by the absence of feedback: the router has no capability metadata to match against and no "ask for clarification" exit, so it guesses and the system pays a full turn per wrong guess.
Recommendation: keep a single supervisor, but give it a bounded label set whose entries are specialist capability descriptions (inputs it needs, tools it may call, what it returns), not open-ended agent names. Add a confidence threshold below which the supervisor routes to a clarification step rather than a specialist. Replace the shared transcript with a supervisor-owned scratchpad so "collaboration" survives without state duplication.
Answering Product: collaboration is preserved through the scratchpad and through hand-back messages; what is removed is untyped broadcasting, which is what generates duplicate tool calls and the bounce cost.
Instrument first: label 500 historical tickets with the correct specialist, measure misroute and duplicate-call rate on that set before changing anything, so the improvement is attributable.
2. Specify the handoff contract and scratchpad schema, and define how you would prove the redesign reduced cost and latency.
Read the worked answer
Handoff contract (enforced with structured/constrained decoding, validated against JSON Schema): intent, specialist, verified_facts (each fact carries value, source tool name, timestamp, record id), open_questions, allowed_tools, turn_budget, trace_id, deadline. The receiving agent may only write facts that came from a tool response - no agent-authored assertions enter verified_facts. That single rule kills most duplicate lookups: the returns agent sees order_id already verified with a timestamp and does not re-call the order service.
Scratchpad is append-only with provenance and a version stamp; compaction summarises closed sub-goals but never rewrites sourced facts.
Proof plan: replay 200 historical tickets through both topologies offline, then a 5% canary for two weeks. Metrics: tool calls per ticket, LLM turns per ticket, misroute rate on the labelled set, cost per ticket, p95 TTFT, and escalation rate. Guardrails on the canary: schema-violation rate must stay under 0.5%, and resolution rate must not regress.
Expect the first-turn count to rise slightly (supervisor validates before dispatching) while total turns and tool calls fall; judge on end-to-end cost, not per-turn latency.
3. Audit wants to know why each ticket went to a specialist. What do you log, and what do you deliberately not log?
Read the worked answer
Log a structured decision record per routing step: candidate specialists with their match scores, the chosen label, a short reason code from a closed vocabulary (for example capability_match, low_confidence_clarify, handback_insufficient_input), the input summary hash, model and prompt version, and the trace id linking downstream tool spans. Emit OpenTelemetry spans per agent step so the record joins to latency and tool calls.
Do not log raw chain-of-thought. It is unstable across model versions, cannot be treated as an accurate account of the computation, and in a support context it will capture customer names, addresses and order contents that then propagate to every log consumer.
Address the original gap by making the reason code mandatory: the supervisor's structured output must include it, and the runtime rejects handoffs without it, so an unexplainable route becomes a validation error rather than a silent audit hole.
Pair this with retention and access rules - decision records kept for the compliance window, raw ticket text kept briefly and access-controlled - and replay the reason-code distribution weekly; a shift in the mix is an early warning that routing behaviour drifted after a prompt or model change.