AI Technologies and Controls
This domain tests how AI systems are built, deployed, and controlled in practice: data handling, model behavior, LLM-specific attack surfaces, and technical controls that remain effective after deployment. Expect scenario questions that contrast a prompt-level instruction with an enforcement-layer control.
Lifecycle and data controls
- Govern data at ingestion: provenance, quality checks, labeling standards, consent, retention, and lineage. Poisoned or unrepresentative training data is a security and safety issue, not just a data-quality issue.
- Separate development, test, and production environments. Restrict training data and model artifacts with least privilege, encryption, and secrets management.
- Maintain an ML bill of materials: base models, datasets, libraries, adapters, and third-party services. Verify signatures and sources before deployment.
- Validate with held-out data, adversarial testing, bias checks, and acceptance thresholds. Record results against intended use.
Model and LLM controls
- Treat model input as untrusted. Use input validation, output encoding, content filters, and rate limits.
- For RAG and agents, enforce authorization at retrieval and tool layers, not only in the prompt. Apply document-level ACLs, least-privilege service accounts, tool allowlists, sandboxing, and human approval for high-impact actions.
- Log prompts, retrieval sources, tool calls, and outputs with privacy safeguards. Monitor drift, abuse, data leakage, and evaluation regressions.
- Common LLM risks include prompt injection (direct and indirect), sensitive information disclosure, excessive agency, and model supply-chain compromise.
Worked example
A support assistant answers from internal wikis. A malicious uploaded document contains hidden text telling the model to reveal another customer's contract. Adding 'ignore malicious instructions' to the system prompt is weak. The stronger design applies retrieval ACLs so the user cannot retrieve unauthorized documents, scans ingested content, restricts tools, and uses output DLP. Reasoning: authorization must be enforced by the data layer; the model is not a security boundary.
Traps
- Relying on prompt wording as an access control.
- Testing only benign prompts.
- Ignoring indirect injection through documents, email, or web pages.
- Giving an agent broad API credentials.
- Monitoring accuracy but not security events or drift.