AI Gateway: Prompt Injection, PII, and Provider Failover
A financial services firm has approved three external LLM providers and built an AI gateway in front of them. Twelve internal applications and 3,000 employees use the gateway; 40% of traffic is employee chat, 60% is application API calls that include customer account summaries. The security team has enabled prompt-injection detection in monitor mode and PII redaction for credit card numbers, but not for account numbers or addresses. A red-team exercise showed that a malicious PDF uploaded to a support-summarization app caused the model to emit another customer's account balance when the retrieved document contained an instruction. The gateway currently logs full prompts and responses for 90 days to a SIEM. The privacy office wants logging reduced to 7 days and redaction extended; the fraud team wants full prompts for investigations. One provider had a 4-hour outage last month, and the failover provider has different data residency and a 2x cost.
1. The red-team success came through a retrieved PDF, not the user prompt. What layered controls would you add, and what would you require before moving prompt-injection detection from monitor to blocking?
Read the worked answer
The attack is indirect prompt injection: untrusted retrieved content overrides the intended instruction. The model cannot reliably distinguish data from instructions, so the fix must be outside the model.
- Enforce retrieval authorization per user and per tenant. The summarization app must not retrieve another customer's document at all; the model cannot leak what it never receives. Use document-level ACLs at the retrieval layer, not a post-hoc filter.
- Treat retrieved text as untrusted: strip or escape instruction-like content, wrap it in delimiters, and use an instruction hierarchy that tells the model to ignore instructions in documents. This reduces but does not eliminate the risk.
- Add output guardrails: DLP for account numbers, balances, and PII; schema validation so the model returns structured fields, not free text; allowlists for tool calls; and a human confirmation step before any action with side effects.
- Add canary tokens and provenance so you can detect when retrieved content is being followed. Log the retrieval document ID with the response for investigation.
- Least privilege for tools: the model should not have direct database or email access; tool calls go through an authorization broker that checks the user's rights.
Before moving from monitor to blocking, require a labeled corpus of benign and malicious documents, per-app false-positive thresholds, a fallback response, and an appeal path. Block only high-confidence detections at first and keep monitor for ambiguous cases. For the support-summarization app specifically, disable free-form balance output and return only fields the user is entitled to see. That control is deterministic and does not depend on detection accuracy.
2. Reconcile the privacy office's 7-day logging and redaction demand with fraud's need for full prompts. Propose a data-handling design.
Read the worked answer
Separate operational telemetry from evidence. The 90-day full-prompt log is a privacy liability and a breach target; the 7-day demand is reasonable for routine operations. Design three tiers:
- Hot operational logs, 7 days: redacted prompts and responses, metadata, model, latency, token counts, and policy decisions. Redact account numbers, addresses, and card data at the gateway before logging, using format-preserving tokenization so support can still correlate.
- Investigation vault, case-based or up to 30 days: full prompt and response available only when a case is opened, with role-based access, justification, dual approval, and immutable audit. Fraud can request a specific session; they do not get a searchable 90-day corpus. Apply legal hold where required.
- Long-term metrics, 90 days or more: aggregates and hashes, no content.
Extend redaction to account numbers and addresses, but do not rely on regex alone; use detectors plus an allowlist for known formats. Tokenize rather than delete where fraud needs to correlate. Route data to the provider region approved for that data class; do not let failover silently move data across borders.
Governance: privacy owns retention, fraud owns investigation criteria, security owns access. Publish a data-flow diagram and a DPIA. The compromise is not 7 versus 90 days; it is minimizing raw content, making re-identification auditable, and giving fraud a lawful, narrow path to the evidence.
3. Design provider failover routing, residency, cost, and graceful degradation for the next outage.
Read the worked answer
Design the gateway as a policy router, not a dumb load balancer. Define approved provider and model pairs per data class and region. Health-check each provider with synthetic requests, track error rate and latency, and use circuit breakers. On primary failure, route only to a provider approved for the same data residency and contract; if none exists, degrade gracefully rather than silently crossing a border.
Graceful degradation by workload: employee chat can fall back to a smaller model or a queue with a user-visible message; customer-facing account summaries should return a safe temporarily-unavailable response or a deterministic template, not a lower-quality model that might hallucinate. Disable non-critical features such as summarization or tone rewriting before core Q&A. Set per-tenant and per-app budget caps and rate limits so a 2x-cost failover does not create a financial incident. Priority tiers keep fraud and support running before marketing experiments.
Test quarterly with a provider outage game day, including DNS, auth, and quota failures. Log the routing decision and provider in response metadata for audit. Review failover cost and residency with legal and finance before the event, not during it. The key judgment is that availability does not outrank residency or customer safety; a controlled degradation is better than an uncontrolled cross-border failover.