Choosing Controls for a Customer-Service Agent With Tool Access

Each proposal targets a different layer, and none alone is sufficient.

• Confirmation prompt before credit (product team): partial. It adds friction but a social-engineered user will simply confirm. It also doesn't address cumulative abuse across a session. Useful as defense-in-depth, not as the primary control.
• Per-session credit cap (fraud team): strong for the credit abuse. The root cause is that the agent could issue unlimited $75 credits with no aggregate limit. A session cap directly bounds the loss. But it must be paired with velocity/aggregate limits across sessions and accounts, or an attacker just opens new sessions.
• Input sanitization (security team): necessary but insufficient. The cross-account data leak came from injected instructions in a customer message reaching an agent with broad read access. Sanitization helps, but the deeper root cause is excessive agency and missing authorization boundaries: the agent could read another account's details at all. The fix is to scope tool calls to the authenticated account, not to filter text.

The 14 bad addresses point to a fourth root cause: no validation of tool-call arguments against account data before execution.

Layer controls by where in the chain the failure occurs: input → model → tool call → outcome.

1. Authorization / least privilege (highest priority)

• Scope every tool call to the authenticated account; the agent should be structurally incapable of reading another account's data.
• Separate read and write permissions; credits and scheduling are write actions requiring stronger checks.

2. Tool-call argument validation (high priority)

• Validate scheduling addresses against the account's known addresses before execution.
• Reject tool calls whose arguments don't match the session's account context.

3. Transaction limits (high priority)

• Per-transaction cap (already $75), per-session cap, and per-account velocity limits over time.
• Route anything above the cap to human approval.

4. Input handling (medium)

• Treat customer content as untrusted; strip/neutralize instruction-like content; keep sanitization as defense-in-depth, not the primary control.

5. Human oversight (medium)

• Confirmation for credits, but designed so it can't be social-engineered into a rubber stamp - show the amount and account, not just 'confirm'.

6. Monitoring (continuous)

• Alert on multiple credits in a session, cross-account access attempts, address mismatches.

Sequencing: ship authorization scoping and argument validation first (they close the data-leak and bad-address root causes), then limits, then input handling and confirmation. Don't ship the confirmation prompt as the headline fix - it creates an illusion of control.

Treat it as a risk-tolerance and design trade-off, not a binary.

• Quantify the legitimate need: what fraction of sessions legitimately involve multiple credits, and what's the distribution? If it's rare, a low cap with human escalation is cheap. If common, a flat cap is the wrong shape.
• Use graduated controls: auto-approve the first credit; the second within a session triggers a step-up (identity check, rep review, or delay); the third requires human approval. This bounds abuse without blocking the common case.
• Segment by risk: new accounts, accounts with recent disputes, or unusual patterns get tighter caps; established accounts get more headroom.
• Make the trade-off explicit and owned: document the residual loss tolerance vs. customer friction, and have the risk owner sign off. Don't let 'it hurts customers' silently override a control without a recorded decision.
• Monitor and tune: track escalation rate and false-positive rate; adjust caps with evidence.

The principle: the cap's purpose is to bound loss, not to forbid multiple credits. Design it to bound loss at the lowest friction that achieves the risk tolerance.