CAIS key terms

A living record of each identified AI risk with its owner, likelihood, impact, existing controls and residual rating, used to prioritise treatment and confirm mitigation work was actually completed.

The discipline of identifying, measuring, mitigating and monitoring the risk that a model produces wrong, unfair or harmful outputs, extended to machine learning systems whose behaviour changes after deployment.

A maintained catalogue of every model, dataset, agent and third-party AI service in use, recording owner, purpose, data sensitivity and deployment status, so controls and incident response can be scoped accurately.

A structured review completed before deployment that examines how an AI system affects individuals and groups, covering rights, fairness, safety and privacy, and records mitigations, owners and formal sign-off.

A designed control in which a competent person reviews, approves or can override AI output, given clear authority, sufficient information and realistic time to intervene before harm occurs.

The amount and type of AI risk an organisation accepts while pursuing its objectives, written as tolerance statements that guide escalation, approval thresholds and go/no-go decisions.

A short standard document stating a model's intended use, training data, evaluation results, limitations and known biases, so downstream teams can judge whether it suits their context.

Assessment of the security, privacy and resilience posture of external model providers, APIs and data suppliers, covering contract terms, subprocessors, change notification and exit plans.

An input deliberately perturbed, often imperceptibly, so a model misclassifies it or returns attacker-chosen output, exploiting the model's sensitivity to particular directions in its input space.

Deliberate corruption of training or fine-tuning data so the resulting model misbehaves on chosen inputs or degrades generally, often entering through scraped, crowdsourced or third-party datasets.

A hidden behaviour implanted during training or fine-tuning that appears only when a specific trigger is present, leaving the model looking normal on ordinary inputs.

An attack in which text in a user prompt or retrieved document makes the model disregard its instructions and follow attacker-supplied ones, potentially triggering tool calls or data disclosure.

A prompt technique that bypasses a model's safety alignment so it produces content its developers intended to refuse, commonly using role-play, encodings or multi-turn framing.

Querying a deployed model's API systematically to reconstruct a functionally similar copy, letting an attacker obtain valuable behaviour without paying training costs.

An attack that decides whether a particular record was used in training by exploiting confidence differences between data the model has and has not seen, revealing a privacy breach.

Reconstructing representative features of training data, such as a recognisable face, from a model's outputs or gradients, exposing information the model was supposed to abstract away.

A design flaw in which an AI agent holds more permissions, tools or autonomy than its task needs, so a confused or manipulated model can take damaging real-world actions.

Information relating to an identified or identifiable person, including identifiers, attributes and inferences drawn about them, which AI systems must minimise, protect and account for.

Collecting and using only the data genuinely needed for a stated purpose and discarding the rest, lowering privacy exposure and shrinking the impact of any breach.

A mathematical guarantee that adding or removing one person's record changes published results only slightly, usually through calibrated noise, limiting what can be inferred about any individual.

Replacing direct identifiers with tokens that can be reversed using a separately stored key, reducing exposure while the data usually remains personal data under privacy law.

A model's tendency to reproduce training samples verbatim or nearly so, especially rare or duplicated records, creating a route for leaking personal or confidential information through outputs.

The documented origin, lineage and transformation history of datasets and models, supporting traceability, licence compliance and investigation of corrupted or unlawfully obtained sources.

Techniques that remove the influence of specific training records from an already trained model so deletion requests can be honoured without retraining from scratch.

A training approach that keeps raw data on local devices or sites and shares only model updates, reducing central data concentration while still needing protection against leaked gradients.

Integrating security controls into the machine learning lifecycle, from data preparation and training through packaging, deployment and monitoring, using automation and shared ownership between data science and security teams.

A controlled repository holding model versions with metadata, owners, approvals and evaluation results, acting as the authoritative record of what exists and what is deployed.

A machine-readable inventory of an AI system's components, including base models, datasets, libraries and licences, used for vulnerability tracking, licence checks and provenance.

Cryptographically signing model weights and packages so consumers can verify the artifact came from a trusted builder and was not altered in storage or transit.

A vulnerability where loading a model file, such as a Python pickle, can execute attacker-supplied code, making untrusted model hubs and downloads a direct code-execution risk.

A pre-built capability to disable, revoke or roll back a model or agent quickly, with tested authority and procedures so it works under incident pressure.

Degradation of a model's performance as live data diverges from training data, which can also weaken security filters tuned to the model's original behaviour.

Collecting logs, metrics and traces of prompts, outputs, tool calls and model versions so behaviour can be reconstructed, anomalies detected and incidents investigated.

Runtime controls that inspect and filter model inputs and outputs, blocking disallowed content, injection patterns or sensitive data before harm or disclosure occurs.

Documented roles, detection thresholds, escalation paths, containment options and communication steps for AI-specific events such as data leakage, harmful output or model compromise.

Structured adversarial testing in which people attempt realistic misuse, prompt attacks and data extraction against an AI system to find weaknesses before adversaries do.

Tamper-evident chronological records of who accessed or changed data, models and configurations, used to prove controls operated and to reconstruct events during investigation.

Ongoing testing and evidence gathering rather than a one-off audit, showing that AI controls keep working as models, data and threats change.