Lead AI Risk Manager key terms

The amount and type of risk from AI systems that an organization is willing to accept in pursuit of its objectives, expressed in measurable thresholds that guide design, deployment, and monitoring decisions.

The acceptable deviation from a stated risk appetite for a specific AI system or use case, often expressed as quantitative limits on error rates, harm frequency, or control failures.

Classifying AI use cases into levels of oversight based on potential severity of harm, autonomy, data sensitivity, and reversibility, so that controls are proportionate to risk.

An AI system whose use can cause significant harm to health, safety, fundamental rights, or critical operations, triggering stricter obligations such as conformity assessment, documentation, and human oversight.

The risk that remains after planned controls and mitigations have been applied, which leadership must accept or reduce before deployment and which must be reassessed after material changes.

The process of selecting and implementing options to modify risk, such as avoiding the use case, reducing likelihood or impact, transferring via contract or insurance, or knowingly accepting it.

A maintained record of identified AI risks, each with owner, cause, potential impact, likelihood, existing controls, treatment plan, and status, used to track and report risk posture over time.

A forward-looking metric that signals rising exposure in an AI process, such as drift frequency, override rate, complaint volume, or control test failure rate, used to trigger review before harm occurs.

A governance structure in which operational teams own AI risk, an independent risk and compliance function oversees it, and internal audit provides assurance to the board.

A cross-functional body with authority to approve, condition, or reject AI use cases, set policy, review risk reports, and escalate issues to executive leadership or the board.

A complete register of AI systems and initiatives in the organization, capturing purpose, owner, data used, model type, affected populations, and risk tier, forming the basis for oversight.

The discipline of identifying, measuring, monitoring, and controlling risks from model error, misuse, or inappropriate reliance, extended for AI to cover data quality, drift, and explainability.

Independent assessment of whether an AI model is conceptually sound, performs as intended on representative data, and is fit for its specific use, conducted by parties not involved in development.

Degradation of model performance when live data diverges from training conditions, whether through changes in inputs, user behavior, or the relationship between inputs and outcomes.

Continuous tracking of an AI system in production for performance, drift, bias, safety incidents, and unexpected behavior, with defined thresholds and escalation paths for corrective action.

Designing AI systems and processes so that competent people can understand, intervene in, override, or stop the system, with clear authority and enough time and information to act.

A control arrangement in which a person reviews and approves each consequential AI output before it takes effect, appropriate where errors could cause serious or irreversible harm.

The tendency of people to over-trust automated recommendations and reduce their own scrutiny, which can make nominal human review an ineffective control unless actively designed against.

The degree to which a person can understand why an AI system produced a given output, enabling challenge, accountability, and effective review by users, auditors, and affected parties.

Providing clear, timely information about an AI system's purpose, capabilities, limitations, data use, and risks to the people who use, are affected by, or oversee it.

The ability of an affected person to challenge a decision or output influenced by an AI system and obtain review by a human with authority to change the outcome.

A structured document describing a model's intended use, training data, evaluation results, limitations, and ethical considerations, supporting informed adoption and downstream risk assessment.

A documented process verifying that an AI system meets specified requirements, such as safety, data governance, and oversight obligations, before market placement or deployment.

Systematic and unfair differences in AI outcomes across groups, arising from data, labeling, feature choice, or optimization objectives, that can cause discriminatory or unequal impact.

A quantified measure of parity in AI outcomes across groups, such as equal selection rates or equal error rates, chosen deliberately because different definitions can conflict.

Structured evaluation of an AI system's performance and outcomes across demographic and other relevant groups, using representative data and disaggregated metrics before and after deployment.

Deliberate manipulation of inputs, prompts, or training data to cause an AI system to misclassify, leak information, or behave unsafely, including evasion, poisoning, and prompt injection.

An attack in which malicious instructions hidden in content processed by a language model cause it to ignore its intended instructions, leak data, or take unauthorized actions.

Structured adversarial testing in which experts attempt to elicit harmful, unsafe, or incorrect AI behavior to surface vulnerabilities before real users or attackers find them.

An event in which an AI system causes or contributes to harm, near-miss, or serious unexpected behavior, which must be logged, investigated, and used to improve controls.

A documented procedure defining how AI incidents are detected, contained, escalated, communicated, and remediated, including roles, decision authority, and timelines for notification.

A pre-built mechanism that lets authorized staff quickly disable or roll back an AI system or its access to downstream actions when it behaves unsafely.

Exposure arising from vendors, foundation models, or data providers, including dependency, opacity, data use, security, and contract gaps, managed through due diligence and ongoing monitoring.

The documented origin, lineage, and processing history of data used to train or operate an AI system, supporting quality, rights, and regulatory defensibility.

Policies and controls covering the collection, quality, access, retention, and lawful use of data in AI systems, including protection of personal and sensitive information.

A structured review of how an AI system collects and uses personal data, identifying privacy risks and mitigations such as minimization, purpose limitation, and retention limits.

The degree to which a designed control actually reduces risk when operated in practice, tested through evidence such as sampling, simulation, or independent challenge.

Allocating oversight effort in proportion to the likelihood and severity of harm, so that low-risk AI uses face lighter requirements while high-risk uses face deeper scrutiny.