Ethical Safeguards and Governance of Agentic AI
Domain 3 - Ethical safeguards and governance of agentic AI. The exam tests controls for excessive agency, indirect prompt injection, data leakage, and accountability when agents act on real systems with real credentials.
Governance as code
Classify each agent by risk and blast radius, register it with owner, model version, tools, data sources, and evaluation results, and require approval before granting new tools or scopes. Keep an immutable audit trail of prompts, tool arguments, results, approvals, and human overrides. Policy should be enforceable configuration, not prose in a wiki.
Defense in depth
NeMo Guardrails input and output rails (Colang flows) handle topic control, jailbreak detection, PII redaction, and fact-checking, but content moderation alone is insufficient for agents. Add least-privilege credentials per agent and per tool, endpoint allowlists, scoped short-lived tokens, spending and rate limits, dry-run modes, sandboxed code execution, and human-in-the-loop approval for irreversible or high-value actions. Treat retrieved documents and tool outputs as untrusted input - a trust boundary, not instructions.
Worked example
An HR agent reads employee records and resumes. A resume contains text instructing the agent to email all records. Controls: the resume is tagged untrusted so injected text cannot alter the plan; the email tool is not in the agent allowlist; PII is redacted in output; exports require human approval and are logged. Reasoning: no single control is sufficient, but injection plus no email tool plus approval means the attack has no path to effect.
Traps
Treating guardrails as moderation only; relying on prompt instructions for safety; giving agents broad standing credentials; logging the final answer but not tool arguments; missing indirect injection through RAG or tool responses; no incident response or rollback for a misbehaving agent.