The agent that runs 40 steps and returns nothing
A research agent answers analyst questions by looping: think, search internal documents, optionally call a calculator, repeat until it decides it can answer. The median run is 6 steps. Eight percent of runs hit a 40-step wall-clock timeout and produce no answer at all. There is no step budget in code - the model decides to stop by emitting a plain text block. Two incidents are under review: an agent re-ran the same failing search 15 times with byte-identical arguments, and another run answered a question using a document that was three years stale, because the retrieval tool ranks by link count rather than recency. Cost per answered question is four times the original estimate, and analysts describe roughly one in five answers as 'confidently wrong on a detail'.
1. Why do runs fail to terminate, and how do you stop the identical-retry loop without making the agent dumber?
Read the worked answer
Termination is currently a model judgement with no contract behind it. The model has no reliable sense of how much of the budget it has spent, so 'I need more evidence' is unopposed by anything except a wall-clock kill that happens after the work is already wasted. Separately, it can't tell that a retry is identical to the last one, so it re-reads the same failing result and draws the same conclusion - the classic loop signature.
Fixes that keep capability:
- A step budget in code, with a graceful ending. Track steps and tokens. At ~80% of budget, inject a system-style notice: 'budget nearly exhausted, produce your best answer now with what you have, citing evidence'. At 100%, force a final answer. A partial, cited answer beats a timeout that returns nothing; the 8% currently produce zero value for full cost.
- Duplicate-call detection. Hash (tool name, normalised arguments). On a repeat, do not execute - return a tool_result saying 'already called with these arguments; result was X'. That converts a loop into information the model can act on.
- A visible plan. Require the agent to maintain a short scratchpad of sub-questions and their status. Models re-search when they can't see what they've already established, and a plan makes progress legible.
- Cap tool output size. Dumping whole documents fills context, pushes earlier evidence out of attention, and provokes re-searching. Return the top passages with document ids, not the document.
Instrument steps-per-run and duplicate-call rate; the duplicate-call rate is your early warning that the loop is degrading.
2. An answer was built on a three-year-old document. Where does the fix belong - the retriever, the prompt, or the output?
Read the worked answer
All three, in that order of importance, because this is a provenance failure rather than a reasoning failure.
Retriever. Ranking by link count is a proxy for popularity, not authority or currency. Surface metadata on every chunk - effective date, superseded flag, doc type, owner - and let the retriever filter or down-weight stale sources. Where a policy has been superseded, the old version should not be retrievable at all. This is the highest-leverage fix: the model can only be as current as what it is handed.
Prompt. Make recency an explicit check rather than a latent hope: instruct the agent to state the effective date of each source it relies on, and to prefer the most recent applicable version when documents conflict. Then require the final answer to cite document ids and dates inline.
Output. Citations make the failure visible and auditable. An answer that reads 'per Policy 4.2 (effective 2022-03), which was superseded in 2024' is self-flagging; the uncited version looked confident and was wrong. Add an abstention path - if no retrieved document actually answers the question, the agent must say so rather than assembling an answer from the nearest passage.
For the 'confidently wrong on a detail' problem specifically, add a verification pass: a fresh-context checker that receives only the answer and the cited spans and must confirm each claim is supported. A second pass in the same context tends to defend the original reasoning.
3. Cost per answered question is four times the estimate. Where is the money going and what would you change first?
Read the worked answer
Two structural multipliers, one of them free to fix.
1. Transcript growth. In the naive loop, every step re-sends the entire history - prior thoughts, every tool call, and every tool result. Total input tokens scale roughly with steps squared. A 6-step median is already ~21 transcript-units of re-sending; the 8% tail at 40 steps is hundreds, and it is exactly the tail that dominates the bill.
2. Unproductive tail. Eight percent of runs burn a full budget and return nothing. That spend produces zero answered questions, so it lands entirely in cost-per-answered-question with no offsetting numerator.
What to change, in order:
- Compact tool results. Keep a structured evidence ledger - doc id, date, the 2–3 relevant sentences - and drop the raw document text from history after it has been used. This is the single largest cut and it also improves attention on the relevant evidence.
- Cache the static prefix. System prompt, tool definitions, and any fixed instructions are identical every step; caching them turns repeated reads into cheap ones. This is pure upside and doesn't change behaviour.
- Route before you loop. Classify the question first; many analyst questions are single-lookup and should never enter the agent at all. A cheap classifier in front of the expensive loop usually beats tuning the loop.
- Budget the tail. Step and token caps turn the 8% into cheap partial answers.
Measure tokens per answered question and per step, not per run - otherwise the tail hides inside the median.