Agentic workflow for invoice processing: when to use an agent vs. a fixed pipeline
An enterprise wants to automate accounts-payable invoice processing. Invoices arrive as PDFs in varied layouts, some scanned. The task is to extract vendor, amount, PO number, and line items, then match against purchase orders and route exceptions to humans. A vendor proposes an autonomous agent that 'reasons' over each invoice, calls extraction tools, searches the PO system, and decides whether to pay, hold, or escalate. Current volume is 8,000 invoices per month, with about 12% requiring human review today. Finance requires an audit trail for every decision and cannot tolerate silent errors on payments. A pilot of the agent approach produced good results on clean digital invoices but occasionally took unexpected tool-call sequences on scanned invoices and once approved a mismatched amount.
1. Should this be built as an autonomous agent or a fixed pipeline with bounded model calls? Justify.
Read the worked answer
This should be a fixed pipeline with bounded model calls, not an autonomous agent, because the task is well-understood, high-stakes, and requires determinism.
- The steps are known: extract fields, validate, match to PO, apply rules, route exceptions. There is no open-ended planning problem that justifies agent autonomy.
- Payments demand determinism: The agent's mismatched-amount approval is the decisive signal. A system that decides its own tool-call sequence can skip or reorder validation. A pipeline enforces that validation always runs.
- Use models where they add value: extraction from varied/scanned layouts is a genuine model task (vision/document extraction). Matching and payment decisions should be deterministic code with explicit rules and thresholds.
- Audit trail: A pipeline logs each stage's inputs and outputs, producing a clean audit record. Agent traces are harder to certify and reason about.
- Exceptions: Route low-confidence extractions and rule failures to humans; that is the 12% review path, and the model's confidence can prioritize the queue.
The judgement is that 'agentic' is not a goal; it is appropriate when the path is unknown. Here the path is known and the cost of deviation is financial.
2. How should the team design confidence handling and human review to meet the audit requirement?
Read the worked answer
- Per-field confidence: extraction returns confidence per field. Low-confidence fields trigger human review rather than guessing.
- Deterministic validation gates: PO match, amount tolerance, duplicate detection, and vendor status run as code. Any failure routes to review with a reason code.
- No silent auto-approval above a risk threshold: set monetary and mismatch thresholds; anything above them requires human sign-off regardless of model confidence.
- Immutable audit log: record the source document, extracted values with confidence, validation results, rule versions, and the human decision. Version the rules so past decisions can be reproduced.
- Feedback loop: human corrections feed back into extraction evaluation so accuracy is measured over time, not assumed.
The design principle: models propose, deterministic rules decide, humans handle exceptions, and every step is logged.