AIGP study materials
AIGP study resources

The Refund Agent With Tool Access

AIGP · Put your knowledge to work in context.

All case studies

The Refund Agent With Tool Access

A retailer deploys an autonomous AI agent that can issue refunds up to $250, change shipping addresses, and cancel orders, using a third-party LLM with tool access. The agent resolves 62% of chats without human handoff; the rest escalate. In the first month, 14 customers received refunds for orders they never placed after the agent misread order IDs, and 3 accounts were canceled after the agent interpreted 'cancel my subscription' as 'cancel my order.' The vendor says the system has 'guardrails' but will not disclose them. Marketing wants to advertise 'AI that fixes problems instantly'; legal wants a disclaimer. The board asks who is accountable when the agent errs.

1. Frame the accountability and transparency questions and recommend a governance position.

Read the worked answer

Accountability: the retailer is accountable as the deployer and the party in privity with the customer. It cannot outsource accountability to the vendor or to the model. The vendor may bear contractual responsibility for defects, but the customer's remedy, the regulator's inquiry, and the reputational harm land on the retailer. The board should adopt a named accountable executive, a RACI that includes product, customer service, legal, security, and finance, and a standing review of agent-caused incidents.

Transparency: customers should know they are interacting with an AI agent, what it can do, and how to reach a human. Marketing's 'fixes problems instantly' claim is risky because it implies reliability the system has not demonstrated. Legal's disclaimer is necessary but not sufficient; a disclaimer does not cure deceptive conduct or a broken refund process.

Recommended position:

  • Publish a plain-language AI-use notice at the start of chat.
  • State the agent's authority limits (e.g., refunds up to $250) and the escalation path.
  • Do not advertise error-free resolution; advertise speed with human backup.
  • Report agent-caused incidents to the board monthly until stability is demonstrated.
  • Require vendor disclosure of guardrails as a condition of continued use.

2. Does the agent trigger regulatory obligations? Assess under the EU AI Act, consumer-protection law, and contract law.

Read the worked answer

EU AI Act: A customer-service chatbot must disclose that the user is interacting with AI unless it is obvious. The agent is probably not high-risk under Annex III on these facts, because it is not making creditworthiness, employment, education, or essential-service eligibility decisions. But if it can deny refunds or cancel accounts in ways that determine access to a service, the analysis should be revisited. The retailer also needs to consider whether the vendor's underlying model is a general-purpose AI model with its own provider obligations.

Consumer-protection law: Misrepresenting the agent's reliability or failing to honor stated refund terms can be an unfair or deceptive practice. The 14 erroneous refunds are a payment/contract problem; the 3 wrongful cancellations are a service-access problem. Both require a remediation path, not just a disclaimer.

Contract law: The agent's acts may bind the retailer. If the agent cancels an order or issues a refund, the customer will reasonably treat that as the retailer's act. The retailer cannot point to the vendor's undisclosed guardrails as a defense to the customer.

Recommendation: Treat disclosure as mandatory, fix the transaction-verification failure, and have legal review the marketing claim before launch. Do not rely on a disclaimer to solve an authority-design problem.

3. Propose authority limits and controls for the agent. What would you allow and prohibit?

Read the worked answer

Allow, with safeguards:

  • Refunds up to a low threshold (e.g., $50) only after matching order ID, customer identity, and payment method through deterministic checks - not LLM inference.
  • Address changes only after identity verification and only before shipment.
  • Order cancellations with explicit confirmation ('You are canceling order #X; is that correct?').

Prohibit or require human approval:

  • Refunds above the threshold.
  • Any refund without a verified order match.
  • Cancellation of accounts or subscriptions; route to a human.
  • Changes to payment instruments.

Controls:

  • Deterministic tool calls: the LLM proposes, but a rules engine validates order ID, amount, and customer identity.
  • Confirmation step: explicit yes/no before irreversible actions.
  • Anomaly detection: flag repeated refunds, mismatched IDs, and unusual timing.
  • Kill switch and rollback: ability to reverse agent-initiated transactions.
  • Logging: full prompt, tool call, and outcome logs; 100% review of refunds for the first 90 days.
  • Vendor condition: disclose guardrails and permit audit, or do not deploy.

The board's accountability question is answered by these controls: a named owner, documented authority limits, and a demonstrated ability to detect and reverse errors.