Security, Governance and Responsible AI
What this domain tests
This domain tests protecting data and using the assistant responsibly: identity and access, secrets, sensitive data, grounding, human oversight and governance evidence.
Core rules
- Use least privilege. Separate roles for builders, publishers and operators; scope API keys and service IDs to the needed resource group.
- Encrypt in transit and at rest; know retention and data-handling settings. Do not assume a feature is compliant because it is hosted.
- Minimize sensitive data in prompts, logs and session variables. Redact where supported, and avoid sending full records when a token or order ID is enough.
- Treat user input and retrieved documents as untrusted. Delimit them in prompts, restrict tool permissions and require confirmation before high-impact actions.
- Keep humans in the loop for consequential decisions: eligibility, money, health, employment and legal outcomes.
- Document intended use, owners, test results, known limitations and fallback routes. Governance is evidence, not a slogan.
Worked example
A benefits assistant answers eligibility questions. It retrieves only public policy, never the user’s case file; it asks for a case ID only when a transactional action needs it; it logs the action and outcome but masks the ID. A “you may be eligible” answer includes a link to a human reviewer and does not make a final determination. Reasoning: the design separates general guidance from individual adjudication, limits data exposure and keeps a human accountable.
Traps
Putting API keys in client-side code; logging full prompts containing PII; treating a prompt instruction as an access control; assuming citations prove correctness; and launching a high-impact flow with no human escalation.