AAISM cheat sheet

Three-lines model for AI: first line (build/operate) owns model risk; second line (risk/compliance) sets AI policy, risk appetite, and independent challenge; third line (internal audit) audits the AI control environment - the same person must not both build and independently assure a model.

Tiered AI inventory rule: every AI system is registered with owner, purpose, data classes, model type, autonomy level, and affected population; classification tiers (e.g., prohibited / high-impact / limited / minimal) drive the depth of assessment, approval authority, and review cadence.

Human oversight is a control, not a sentiment: define who can override, on what evidence, within what time window, and log every override with actor, timestamp, model version, and rationale.

Regulatory map (know the trigger, not just the name): EU AI Act - risk-tiered obligations, GPAI transparency, prohibited practices from Feb 2025, high-risk obligations phased from Aug 2025–Aug 2027; NIST AI RMF - Govern, Map, Measure, Manage, voluntary; ISO/IEC 42001 - certifiable AI management system; ISO/IEC 23894 - AI risk guidance; OECD AI Principles - human-centred values, transparency, accountability.

Model lifecycle gate rule: no model reaches production without documented sign-off at each gate - intake/classification, data provenance review, development testing, independent validation, deployment approval, post-deployment monitoring, and decommissioning.

Accountable executive: assign a named AI accountable owner per system plus a program-level AI governance council with decision rights for risk acceptance above threshold; risk acceptance must be time-boxed and re-approved.

Third-party and foundation-model rule: vendor AI (including embedded AI in SaaS) is in scope - require model cards or equivalent documentation, training-data provenance statements, evaluation results, incident-notification SLAs, and contractual rights to audit and to change/exit.

Governance metrics that auditors look for: percentage of AI systems inventoried and tiered, percentage with completed impact assessments, overdue control remediation count, override and incident rates, and drift-alert resolution time.

Supervised learning minimizes a loss over labelled pairs (x, y); unsupervised learning finds structure without labels (clustering, dimensionality reduction); reinforcement learning optimizes a policy against a reward signal - reward misspecification is a primary control concern.

Overfitting rule: low training error with high validation error signals overfitting; controls are held-out validation, cross-validation, regularization, and a documented train/validation/test split with no leakage between them.

Data-poisoning control: verify provenance and integrity of training data, restrict write access to training pipelines, hash and version datasets, and monitor for anomalous label distributions before retraining.

Prompt injection is an input-validation and privilege problem, not a model problem: treat model output as untrusted, separate instructions from data, apply least-privilege tool access, and require confirmation for consequential actions.

RAG (retrieval-augmented generation) controls: access-control-filtered retrieval so the model cannot surface documents the user lacks rights to; source citation; freshness checks on the index; and logging of retrieved chunks for audit.

Model extraction and inversion defenses: rate-limit and monitor query patterns, avoid returning raw confidence vectors where not needed, add output perturbation or watermarking, and restrict API access by contract and authentication.

Evaluation control set: task accuracy/F1, calibration, robustness to distribution shift and adversarial inputs, bias testing across protected groups, hallucination/groundedness rate, latency and cost, and red-team findings with remediation status.

MLOps integrity controls: version control for code, data, and model artifacts; reproducible training environments; immutable model registry entries; signed deployments; and rollback capability with tested restoration.

Risk formula: inherent risk = likelihood × impact assessed before controls; residual risk = inherent risk reduced by control effectiveness; risk is accepted only when residual ≤ appetite, with named acceptor and expiry date.

NIST AI RMF function sequence: Govern (culture, accountability, policies) → Map (context, stakeholders, impacts) → Measure (metrics, testing, tracking) → Manage (prioritize, respond, monitor); Govern is continuous and wraps the other three.

AI-specific harm categories to enumerate: bias and unfair discrimination, privacy and re-identification, security (poisoning, evasion, extraction, injection), safety and physical harm, hallucination/misinformation, IP and copyright, third-party and concentration risk, and environmental/cost risk.

Fundamental rights / impact assessment rule: high-impact uses (employment, credit, education, health, essential services, law enforcement, migration) require a documented impact assessment covering affected groups, redress mechanism, and human review path before deployment.

Bias testing method: define protected attributes and outcome metrics, compare error and selection rates across groups, examine proxy variables and label bias in training data, and record thresholds and remediation decisions - fairness metric choice is a policy decision, not a purely technical one.

Monitoring and drift rule: track data drift (input distribution change), concept drift (input–outcome relationship change), and performance decay; define alert thresholds, owners, and retraining or suspension triggers in advance.

Incident response for AI extends standard IR: detect (monitoring, user reports), contain (disable model, revoke keys, roll back version), assess harm and notification duties, remediate, and feed findings back into risk assessment and control design.

Concentration and dependency risk: single-model or single-vendor dependence, upstream foundation-model changes, and data-supplier dependence are risks in their own right and belong in the enterprise risk register with exit and fallback plans.