CAIS cheat sheet

NIST AI RMF defines four functions - GOVERN, MAP, MEASURE, MANAGE. GOVERN is cross-cutting and is not a sequential lifecycle stage; it sets accountability, culture and risk tolerance.

EU AI Act prohibited practices (in force since February 2025) include social scoring by public authorities, subliminal or manipulative techniques causing harm, untargeted facial-image scraping, emotion recognition in workplaces and schools, and most real-time remote biometric identification in public spaces.

EU AI Act timing: prohibitions from 2 February 2025, general-purpose AI obligations from 2 August 2025, most high-risk obligations from 2 August 2026, and Annex I product-embedded systems from 2 August 2027.

ISO/IEC 42001 is a certifiable AI management system standard structured on Plan-Do-Check-Act; ISO/IEC 23894 provides AI risk management guidance and is guidance only, not a certifiable management system.

Accountability rule: every AI system carries a named business owner, a named technical owner and a documented risk-acceptance authority; the governing body approves the AI risk appetite and reviews the AI inventory at least annually.

EU AI Act Article 27 requires certain high-risk deployers - notably public bodies and private entities providing public services or creditworthiness assessments - to complete a fundamental rights impact assessment before first use.

Three lines model applied to AI: product and engineering own and operate controls (first line), independent risk and compliance set limits and challenge (second line), internal audit provides assurance to the board (third line).

EU AI Act systemic-risk presumption for general-purpose models is training compute above 10^25 FLOP; those providers must perform model evaluation, adversarial testing, systemic-risk assessment and serious-incident reporting.

Model cards document intended use, training data, evaluation metrics and limitations; system cards extend coverage to the full application, its guardrails, retrieval sources and downstream integrations.

NIST AI RMF trustworthiness characteristics (seven): valid and reliable; safe; secure and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; fair with harmful bias managed.

Risk score = likelihood × impact. Scales, thresholds and escalation levels must be defined in policy before any assessment begins; choosing them per system destroys comparability across the portfolio.

Risk treatment hierarchy for AI: mitigate with a control, transfer via insurance or contract, avoid by not deploying, or accept with documented sign-off at the level matching the residual rating.

Policy hierarchy: policy (board-approved intent) → standard (mandatory control baseline) → procedure (step-by-step implementation) → guideline (advisory). A control with no owner and no test is not a control.

Inherent risk is assessed before controls; residual risk after. For the residual calculation, treat untested or unmonitored mitigations as absent until evidence of operating effectiveness exists.

AI risk taxonomy baseline: harmful bias and discrimination, hallucination and misinformation, privacy and memorisation, security compromise, IP and copyright, third-party dependency, model drift, cost/availability, overreliance, regulatory change.

Third-party AI due diligence must cover training-data provenance and rights, model and weights provenance, subprocessor list, breach and model-change notification timelines, evaluation/red-team evidence, and exit with data deletion.

Risk appetite is expressed as tolerance thresholds by tier - for example, no high-risk system may go live with an open high-severity finding, and any accepted risk above medium requires sign-off one level above the system owner.

Controls should be mapped across ISO/IEC 27001 Annex A, NIST SP 800-53 and ISO/IEC 42001 Annex A so one control test satisfies multiple frameworks and the register shows coverage gaps explicitly.

NIST AI 100-2 taxonomy splits attacks by adversary goal and stage: evasion (inference-time), poisoning (training-time), and privacy attacks - membership inference, model inversion, attribute inference and model extraction.

Evasion methods: FGSM (single-step gradient sign), PGD (iterative), C&W (optimisation-based). Perturbations are bounded by an Lp norm; black-box attacks succeed because adversarial examples transfer between models.

Poisoning variants: availability poisoning degrades overall accuracy; targeted/backdoor poisoning binds a trigger pattern to a chosen output; clean-label poisoning keeps labels correct so manual review does not catch it.

Poisoning defences: source allowlisting, dataset hashing and versioning, near-duplicate removal, embedding-cluster outlier detection, label QA sampling, trigger reverse-engineering, and retraining from a verified snapshot after detection.

Privacy attacks: membership inference exploits overconfidence on training records; model inversion reconstructs representative inputs; model extraction clones functionality through the query API and is as much an IP threat as a security one.

OWASP Top 10 for LLM Applications (2025): prompt injection, sensitive information disclosure, supply chain, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, unbounded consumption.

Prompt injection is not fixed by prompt wording. Direct injection comes from the user, indirect injection from retrieved documents, web pages or tool output; mitigations are privilege separation, tool allowlisting, output validation and human approval for irreversible actions.

MITRE ATLAS mirrors ATT&CK for AI systems, with tactics spanning reconnaissance, resource development, initial access, ML model access, execution, persistence, defence evasion, discovery, collection, exfiltration and impact.

Adversarial defences that work: adversarial training, input pre-processing, randomized smoothing, certified robustness. Gradient masking and obfuscation are not defences - they break under transfer and adaptive attacks.

Classify training, validation, fine-tuning and retrieval data into public / internal / confidential / restricted; the classification flows down to derived artefacts - weights, embeddings, logs, evaluation sets and backups.

Encryption baseline: AES-256 at rest, TLS 1.2 or higher in transit, keys in a KMS or HSM, separate keys per environment, and documented rotation. Model weights inherit the classification of their most sensitive training data.

Access control: RBAC or ABAC with least privilege, MFA on data lakes and model stores, break-glass access that is logged and reviewed, and quarterly entitlement recertification.

Ingestion controls against poisoning and contamination: source allowlisting, hash verification of downloaded datasets, near-duplicate detection, label QA sampling, and anomaly detection over embedding clusters.

Immutability rule: any change to a dataset produces a new version with a new content hash and triggers a documented retraining decision; do not patch datasets in place.

Vector and RAG stores contain raw chunk text and embeddings that can be approximately inverted. Treat them as confidential data repositories with tenant isolation, encryption, access logging and retention limits.

Deletion reality: exact removal of data from trained weights requires retraining or machine unlearning; crypto-shredding (destroying the per-dataset encryption key) is the practical control for backups and cold storage.

DLP and egress control on pipelines and notebooks: block copying restricted data to public SaaS or external LLM endpoints, restrict outbound network access from training jobs, and log all data exports.

Secrets hygiene: no credentials in notebooks, repositories or prompts; use a vault with short-lived tokens, separate service identities per pipeline stage, and automated secret scanning on commits.

GDPR lawful bases (six): consent, contract performance, legal obligation, vital interests, public task, legitimate interests. Special-category data additionally needs an Article 9 condition, and legitimate interests cannot be used by public authorities for their tasks.

A DPIA is required for high-risk processing - large-scale special-category data, systematic monitoring of public areas, profiling with significant effects, or innovative technology. If residual risk stays high, the supervisory authority must be consulted before processing.

Data subject rights to operationalise in AI systems: access, rectification, erasure, restriction, portability, objection, and the Article 22 right not to be subject to solely automated decisions with significant effects.

Core principles: lawfulness, fairness and transparency; purpose limitation; data minimisation; accuracy; storage limitation; integrity and confidentiality; accountability with demonstrable evidence.

Privacy-enhancing technologies: differential privacy (epsilon budget), federated learning (combine with secure aggregation or DP because gradients leak), secure multi-party computation, homomorphic encryption, and synthetic data validated for fidelity and membership leakage.

Pseudonymised data is still personal data. Anonymisation must be irreversible against linkage and inference attacks; k-anonymity alone fails against attackers with auxiliary information - add l-diversity or differential privacy.

Fairness metrics - demographic parity, equalized odds, predictive parity - cannot generally be satisfied simultaneously when base rates differ. Selecting the metric is a documented policy decision tied to the use case.

Responsible training data: documented consent or licence, terms-of-service and robots.txt compliance, honoured opt-outs, copyright and IP indemnity review, and a register of sources that can be withdrawn.

International transfers need an adequacy decision, Standard Contractual Clauses or Binding Corporate Rules, plus a transfer impact assessment; NIST Privacy Framework functions are Identify-P, Govern-P, Control-P, Communicate-P and Protect-P.

Threat-model each component with STRIDE plus AI-specific cases (poisoning, model theft, prompt injection, unsafe deserialisation, excessive agency) at design time, and re-run the model whenever a data source, tool or model is added.

Supply-chain controls: pinned dependencies, SCA scanning, SBOM plus an AI bill of materials covering datasets, pretrained weights and licences, and artefact signing with signature verification before deployment (Sigstore/cosign).

Never load untrusted pickle-format models - unpickling executes arbitrary code. Prefer safetensors or ONNX, scan artefacts, and load in a sandboxed environment with restricted network and file-system access.

LLM application rule: model output is untrusted input to every downstream system. Encode and validate it, parameterise database queries, never eval model text, and sandbox any code execution the model can trigger (OWASP LLM05).

Prompt-injection defence in depth: separate the instruction channel from the data channel, label retrieved content as untrusted, allowlist callable tools, use least-privilege tokens per tool, and require human approval for financial, destructive or external-communication actions.

Guardrail stack: input filters for jailbreak patterns and PII, output filters for DLP and harmful content, token and rate limits, schema-constrained structured output, and fail-closed behaviour when a filter errors.

CI/CD security: branch protection and peer review, SAST/DAST/IaC scanning, signed commits, ephemeral build agents, least-privilege pipeline service accounts, and no production data in development environments.

Human oversight is a control, not a formality: high-risk decisions need a defined reviewer role, a documented review record, and a way to override or reject the model output that is auditable.

Least agency for agents: grant only the tools, scopes and data the task requires, scope credentials to a single task, expire them quickly, and log every tool call with its arguments and result.

Lifecycle stages with entry and exit criteria: problem framing, data acquisition and preparation, training and tuning, evaluation and validation, deployment, monitoring, and retirement - each stage produces a signed, retained artefact.

Model registry controls: immutable versioned artefacts with hash, signature, owner and approval status. Production deployments reference a digest, never a mutable tag such as latest.

Environment isolation: separate accounts or clusters for development, staging and production, no production data in development, network policies restricting egress, and a distinct least-privilege role per pipeline stage.

Drift types: covariate (input distribution) drift, concept drift (input-to-output relationship), label drift, and prediction drift. Monitor with distribution tests such as PSI or KL divergence plus delayed ground-truth accuracy.

Retraining triggers: scheduled cadence, drift threshold breach, accuracy below SLA, upstream data-source change, new threat intelligence, or a security or privacy incident involving the model.

Release pattern: shadow deployment (parallel scoring, no user impact) → canary release on a small traffic slice → progressive rollout, with automatic rollback on error-rate, latency or quality SLO breach.

Reproducibility as a security property: pinned dependencies, containerised environments, seeded randomness, versioned data snapshots - without it you cannot investigate tampering or roll back to a known-good model.

Decommissioning checklist: revoke API keys and service principals, remove from the registry, archive or delete artefacts per retention policy, notify downstream consumers, and record the retirement in the AI inventory.

Notebook risk: notebooks execute arbitrary code and frequently hold embedded secrets and data samples. Sanitise outputs before commit, restrict execution identity, and never run untrusted notebooks with production credentials.

Log at minimum: prompts, completions, retrieved document identifiers, tool calls with arguments and results, model version digest, user and session identifiers, latency and token counts - with PII redaction and tamper-evident storage.

Detection signals for AI systems: spikes in refusals or jailbreak patterns, abnormal token consumption or egress volume, unexpected tool invocations, new embedding-cluster outliers, drift breaches, and authentication failures against model APIs.

Assurance KPIs: MTTD, MTTR, false positive rate, percentage of traffic on approved model versions, open high-severity findings, red-team findings closed within SLA, and count of drift threshold breaches per model.

NIST SP 800-61 incident phases: preparation; detection and analysis; containment, eradication and recovery; post-incident activity. AI incidents add model rollback and guardrail changes to the containment step.

AI-specific containment actions: disable agent tools and plugins, rotate all keys and tokens the model could reach, roll back to the last approved model version, block the offending input pattern at the gateway, and restrict egress.

Evidence preservation before remediation: snapshot logs, prompts and completions, model artefact hashes, dataset versions and configuration, and maintain chain of custody - retraining destroys the state you need to investigate.

EU AI Act serious-incident reporting to market surveillance authorities: 15 days general, 2 days for widespread infringement or serious and irreversible disruption of critical infrastructure, 10 days where a person has died.

Assurance sources: NIST AI 600-1 generative AI profile, AI red teaming that includes human adversaries and safety/misuse testing, control testing evidence, internal audit, and third-party certification such as ISO/IEC 42001 or SOC 2.

Post-incident closure: update the risk register, convert the incident into permanent evaluation and regression test cases, add or tighten guardrails, and run a tabletop exercise covering the failure mode discovered.