AI Security Architecture and Controls
Domain 3 tests security architecture for AI systems: how to layer preventive, detective, and responsive controls across data, model, application, API, and agent/tool layers. Expect design-review questions and control-selection scenarios, not pure theory.
Core rules and concepts
- Apply zero trust to AI: authenticate users and services, authorize each model/API/tool call, and do not trust model output by default.
- Segment the AI stack: training environment, model registry, inference service, vector store, tool/plugin layer, and logging pipeline.
- Enforce least privilege for model access, retrieval scopes, and agent tools. A RAG assistant should retrieve only documents the requesting user may see.
- Treat prompts and model output as untrusted input. Use input validation, output encoding, content filters, and guardrails, but do not rely on a single filter.
- Protect the supply chain: approved model sources, artifact signing, dependency scanning, ML-BOM/SBOM, and reproducible deployments.
- Log prompts, retrievals, tool calls, and outputs with privacy controls; keep secrets out of prompts and logs.
Worked example
An LLM assistant can query a CRM and send email. Architecture: API gateway authenticates users; an authorization service issues short-lived tokens scoped to CRM records; the retrieval layer enforces record-level access; tool calls require explicit user confirmation and allow-listed actions; egress filtering blocks unknown domains; output is logged with PII redaction. The model cannot hold standing credentials. If indirect prompt injection asks it to email customer data externally, the tool policy denies the action and the attempt is logged.
Traps that cost marks
- Relying on a perimeter firewall while the model has broad API and tool access.
- Giving an agent standing credentials or unrestricted internet egress.
- Trusting retrieved content as safe because it is internal.
- Logging full prompts with secrets and personal data.
- Assuming a vendor model is secure without contractual and technical assurance.