AI-RRS key terms

The potential for harm or loss arising when a model produces inaccurate, misleading, or misapplied outputs, including losses from decisions made using those outputs.

Systematic and repeatable errors in an automated system that create unfair outcomes for particular groups, often arising from skewed training data, flawed features, or proxy variables.

The degree to which the reasoning behind a model's output can be understood and communicated to stakeholders such as auditors, regulators, customers, and decision-makers.

A model whose internal logic is not readily interpretable by humans, so its behavior can only be assessed through inputs, outputs, and post-hoc explanation techniques.

A change over time in the statistical relationship between model inputs and the target outcome, degrading model performance even when input distributions look stable.

A shift in the distribution of input data compared with the data used to train or validate a model, which can undermine the reliability of its predictions.

A generated output that is fluent and confident but factually wrong or unsupported by source material, common in large language models.

Exposures specific to systems that create text, images, or other content, including fabricated facts, harmful content, intellectual property issues, and leakage of sensitive information.

An attack in which crafted input causes a language model to ignore its intended instructions and follow the attacker's, potentially exposing data or triggering unauthorized actions.

A maintained register of an organization's models recording purpose, owner, risk rating, dependencies, and status, serving as the foundation for governance and oversight.

The accountable individual or function responsible for a model's design, performance, documentation, and remediation of issues throughout its lifecycle.

A governance structure in which operational teams manage risk, an independent risk function provides oversight, and internal audit gives assurance.

A structured set of policies, roles, processes, and controls that directs how an organization develops, deploys, and supervises AI systems in line with its risk appetite.

The amount and type of risk an organization is willing to accept in pursuit of its objectives, expressed as statements or limits that guide AI decisions.

The acceptable variation around a risk appetite, often expressed as concrete thresholds or limits that trigger escalation when breached.

A voluntary U.S. framework organizing AI risk work into govern, map, measure, and manage functions to help organizations build trustworthy AI systems.

An international standard specifying requirements for an AI management system, covering governance, risk treatment, and continual improvement within an organization.

A structured process of identifying, analyzing, and evaluating risks by considering likelihood and impact to prioritize treatment and allocate resources.

A living record of identified risks with owners, ratings, controls, and treatment plans, used to track and report risk status over time.

The level of risk that exists before any controls or mitigations are applied, providing a baseline for judging the value of controls.

The risk remaining after existing controls and mitigations have been applied, which must be compared against risk tolerance.

Independent evaluation of whether a model is conceptually sound, performs as intended, and is appropriate for its use, typically before deployment and periodically thereafter.

Assessment of a model or AI system performed by parties not responsible for building it, to reduce conflicts of interest and provide objective challenge.

Deliberate probing of a system with crafted or edge-case inputs to reveal weaknesses, unsafe behavior, or exploitable vulnerabilities before and after deployment.

A structured exercise in which a team simulates attackers or adversaries to find failures, misuse paths, and safety gaps in an AI system.

Arrangements ensuring people can monitor, intervene in, or override an AI system's outputs, especially for high-impact decisions.

A control design in which a person reviews and approves an AI output before it takes effect or affects a customer or decision.

The tendency of people to over-trust automated recommendations and reduce their own scrutiny, weakening the effectiveness of human review controls.

A control strategy layering multiple independent safeguards so that failure of one control does not leave the system unprotected.

Actions taken to reduce the likelihood or impact of a risk, such as redesigning a model, adding controls, restricting use, or transferring risk.

Ongoing tracking of a deployed model's performance, inputs, and outputs to detect degradation, drift, or unexpected behavior and trigger remediation.

A metric chosen to give early warning that risk exposure is rising, such as error rates, complaint volumes, or override frequencies, tied to escalation thresholds.

Quantitative measures comparing model outcomes or error rates across groups, used to detect and track disparate treatment or impact.

A situation where a neutral-seeming practice or model produces substantially different outcomes for a protected group, a concept central to fair-lending and employment scrutiny.

Written records describing a model's purpose, data, assumptions, limitations, and validation results, supporting auditability and informed use.

A chronological record of system activity, inputs, outputs, and changes that allows reconstruction of decisions and supports investigation and regulatory review.