Retrieved text asks an assistant to exceed its authority
A fictional AI application retrieves a document to summarize it for a user. The document contains text instructing the assistant to ignore its task and send private records to a new external address. The application has a tool capable of sending data, but the user requested only a summary and no approval for that transmission has occurred. A colleague proposes treating the document’s request as a higher-priority instruction because retrieval was successful. Another treats it as untrusted content and relies on application authorization, limited tool rights and meaningful review of consequential actions. This written exercise sends nothing and calls no model or tool. It asks which trust boundary is involved and why a model’s own assertion that permission exists is insufficient.
1. What is the relevant boundary?
Read the worked answer
Retrieved content is data for the user’s task, not independent authority to disclose private records. Its embedded request is an indirect instruction attempt outside the authorized summary scope.
2. Which controls matter?
Read the worked answer
Use enforced tool/resource permissions, validated destinations and actual meaningful authorization for consequential actions. Do not accept the model’s claim that approval occurred as the approval mechanism.