Agent Workflow Hits Tool-Call Loops Under Load
You built a generative-AI application where an agent answers operations questions by calling three tools: a metrics query API, a runbook search, and a ticket-creation API. In staging it works. In production at 40 concurrent users, you see: 8% of sessions exceed 12 tool calls (staging max was 4), p95 session time is 90 s, the ticket API has created 60 duplicate tickets in a day, and cost per session tripled. Logs show the agent re-querying metrics with slightly different time windows and retrying ticket creation after slow responses.
1. Explain the duplicate tickets and the call explosion.
Read the worked answer
Two distinct failure modes, both triggered by production conditions:
- Duplicates: the ticket API is not idempotent from the agent's perspective. Under load, responses are slow; the agent (or the orchestration layer's retry policy) times out and retries, and each retry creates a new ticket. Staging was fast enough that retries never fired. Fix: idempotency keys per intended action, and a retry policy that distinguishes safe-to-retry (reads) from unsafe (writes).
- Call explosion: the metrics tool returns results the agent interprets as insufficient, so it re-queries with nudged parameters - a loop with no stopping criterion. Under load, latency and partial results increase, which the agent misreads as "try again." Cap tool calls per session, require an explicit reason to re-query the same tool, and return clearer "no more data" signals from the metrics API.
Both are orchestration bugs, not model-quality bugs.
2. Design the guardrails you would add.
Read the worked answer
Layer them:
- Per-session tool-call budget (e.g., 8) with a hard stop that returns a partial answer plus "needs human follow-up" rather than looping.
- Idempotency keys on ticket creation derived from session + intent, so retries are deduplicated server-side.
- Retry policy split: reads retry with backoff; writes never auto-retry - surface a confirmation or a queued action instead.
- Loop detection: if the same tool is called with near-identical arguments twice, block the third and force the agent to either use the result or escalate.
- Time budget per session (e.g., 45 s) so p95 cannot silently drift to 90 s.
These convert unbounded agent behavior into bounded behavior with explicit degradation.
3. How do you reproduce and verify the fix, given staging never showed it?
Read the worked answer
Staging didn't reproduce it because the trigger is load-dependent latency and partial results. So the test harness must inject those conditions:
- Fault injection: add artificial 2–5 s latency and intermittent partial responses to the metrics and ticket tools in staging, then run 40 concurrent synthetic sessions.
- Assertions: no session exceeds the call budget; duplicate tickets = 0 under injected retries; p95 session time under budget; escalation path fires when the budget is hit.
- Replay the 60-duplicate day's logs against the new orchestration to confirm the idempotency key collapses them to one.
Also add a canary at 10% production traffic with duplicate-ticket and call-count dashboards before full rollout.