AAISM key terms
AAISM · Get comfortable with the vocabulary.
- AI Management System (AIMS)
- Related: AI Policy, Control Objective, Continual Improvement
A structured set of policies, roles, processes, and records an organization uses to govern AI across its lifecycle, typically aligned to a recognized standard so that objectives, controls, and continual improvement are auditable.
- AI Governance
- Related: Three Lines Model, AI Risk Appetite, Accountable Executive
The decision rights, accountability structures, and oversight mechanisms that determine how an organization selects, builds, deploys, and retires AI systems, including who may approve risk acceptance and how exceptions are escalated.
- AI Policy
- Related: AI Management System (AIMS), Acceptable Use Policy, Use Case Inventory
A formal management-approved statement defining permitted and prohibited AI uses, mandatory review gates, and consequences for noncompliance, forming the top-level requirement that lower-level standards and procedures must satisfy.
- AI Risk Appetite
- Related: AI Governance, Residual Risk, Risk Tolerance Threshold
The amount and type of AI-related risk an organization is willing to accept in pursuit of its objectives, expressed in qualitative or quantitative terms and used to decide which use cases need mitigation, escalation, or rejection.
- Three Lines Model
- Related: AI Governance, Internal Audit Assurance, Control Testing
An accountability structure in which operational teams own and manage AI risk, an independent risk and compliance function provides oversight, and internal audit gives assurance to the governing body on the design and effectiveness of controls.
- Use Case Inventory
- Related: AI Policy, Model Card, Shadow AI
A maintained register of every AI system and generative feature in use or planned, recording owner, purpose, data involved, model or vendor, risk rating, and approval status so that governance activities can be scoped and prioritized.
- Shadow AI
- Related: Use Case Inventory, Acceptable Use Policy, Data Loss Prevention
Use of AI tools, APIs, or browser extensions by staff without organizational approval or visibility, creating unmanaged data exposure, licensing, and compliance risk that inventories and network monitoring are designed to detect.
- Acceptable Use Policy
- Related: Shadow AI, AI Policy, Data Loss Prevention
Rules telling staff which AI tools and inputs are allowed, what data may never be entered into external models, and what disclosure or attribution is required, typically paired with training and monitoring for enforcement.
- AI Governance Committee
- Related: AI Risk Appetite, Use Case Inventory, Accountable Executive
A cross-functional body with delegated authority to approve AI use cases, set risk thresholds, resolve escalated issues, and review portfolio-level risk reporting, usually including legal, risk, security, data, and business representation.
- Accountable Executive
- Related: AI Governance, Residual Risk, AI Governance Committee
The named senior leader who owns an AI system's outcomes and formally accepts its residual risk, ensuring a single point of accountability rather than diffuse ownership across teams.
- Model Card
- Related: System Card, Use Case Inventory, Benchmark Evaluation
A short structured document describing a model's intended use, training data characteristics, evaluation results, known limitations, and ethical considerations, used to inform reviewers and downstream consumers before adoption.
- System Card
- Related: Model Card, Guardrail, Red Teaming
Documentation for a deployed AI product that describes its architecture, safeguards, evaluated risk areas, and known failure modes at the system level, complementing model-level documentation with deployment context.
- Supervised Learning
- Related: Training Data, Overfitting, Benchmark Evaluation
A training approach that fits a model to labeled input-output pairs so it can predict labels or values for new inputs, with performance bounded by label quality and by how well training data represents production data.
- Unsupervised Learning
- Related: Anomaly Detection, Training Data, Feature Engineering
A training approach that finds structure in unlabeled data, such as clusters, density estimates, or lower-dimensional representations, often used for segmentation, anomaly detection, and exploratory analysis rather than prediction of a known target.
- Reinforcement Learning
- Related: Reward Hacking, Human Feedback Alignment, Agentic AI
A training approach in which an agent learns behavior by taking actions and receiving rewards or penalties, optimizing a policy that can be difficult to specify and that may exploit unintended reward shortcuts.
- Large Language Model (LLM)
- Related: Hallucination, Prompt Injection, Retrieval-Augmented Generation
A neural network trained on large text corpora to predict and generate language, used for drafting, summarization, retrieval-augmented answering, and code assistance, and prone to fluent but incorrect output.
- Hallucination
- Related: Retrieval-Augmented Generation, Human in the Loop, Grounding
Confidently stated model output that is unsupported by evidence or factually wrong, arising from statistical generation rather than lookup, and mitigated through grounding, citation requirements, and human verification rather than prompt wording alone.
- Retrieval-Augmented Generation
- Related: Grounding, Hallucination, Vector Database
A pattern in which relevant documents are retrieved from an approved source and supplied to a model at query time so answers are grounded in that content, reducing unsupported output while introducing dependence on retrieval quality and access controls.
- Grounding
- Related: Retrieval-Augmented Generation, Hallucination, Guardrail
Constraining model output to verifiable sources such as retrieved documents, databases, or tool results, so that claims can be traced and checked and unsupported assertions are flagged or suppressed.
- Prompt Injection
- Related: Indirect Prompt Injection, Agentic AI, Guardrail
An attack in which adversarial text in a user prompt or in retrieved content causes a model to ignore its instructions or perform unintended actions, a serious concern when the model can call tools or access data.
- Indirect Prompt Injection
- Related: Prompt Injection, Retrieval-Augmented Generation, Agentic AI
An attack that hides malicious instructions in content the model will later process, such as a web page, email, or document, so the payload reaches the model without the user typing it.
- Guardrail
- Related: Prompt Injection, Human in the Loop, Red Teaming
A runtime control that constrains model behavior, such as input filtering, output classification, allow-listed tool calls, or refusal rules, intended to reduce harmful or noncompliant outputs but never to replace human review for high-impact decisions.
- Fine-Tuning
- Related: Training Data, Overfitting, Model Drift
Further training of a pretrained model on task- or domain-specific data to adapt its behavior, which can improve accuracy but may embed sensitive data, erode prior safety behavior, and complicate version traceability.
- Training Data
- Related: Data Provenance, Overfitting, Bias
The dataset used to fit model parameters, whose provenance, licensing, representativeness, labeling quality, and embedded personal or confidential information determine many downstream accuracy, fairness, and legal risks.
- Overfitting
- Related: Training Data, Model Drift, Benchmark Evaluation
A modeling failure in which a model learns patterns specific to its training sample, producing strong training performance but poor generalization to new data and inflated confidence in production.
- Model Drift
- Related: Overfitting, Continuous Monitoring, Benchmark Evaluation
Degradation of model performance over time as production inputs, user behavior, or the underlying relationship between inputs and outcomes change, detected through ongoing monitoring of data distributions and outcome metrics.
- Explainability
- Related: Interpretability, Model Risk, Adverse Action Notice
The degree to which the reasons for a model's output can be communicated to reviewers, affected parties, and regulators in understandable terms, supported by techniques such as feature attribution, surrogate models, and example-based rationales.
- Interpretability
- Related: Explainability, Model Risk, Benchmark Evaluation
The extent to which a human can understand the internal logic of a model, which is often higher for simpler models and lower for large neural networks, and which shapes what assurance and challenge are realistically possible.
- Bias
- Related: Fairness Testing, Training Data, Protected Attribute
Systematic error that produces unfair or skewed outcomes for particular groups, arising from unrepresentative data, flawed labels, proxy variables, or objective choices, and assessed through disaggregated performance testing.
- Protected Attribute
- Related: Bias, Fairness Testing, Disparate Impact
A characteristic such as race, sex, age, disability, or religion that is legally shielded from discriminatory treatment, requiring careful handling in data collection, feature use, and outcome testing for AI systems.
- Disparate Impact
- Related: Protected Attribute, Fairness Testing, Bias
A statistically measurable difference in outcomes across groups produced by a neutral-seeming rule or model, which can create legal exposure even without discriminatory intent, and which testing is designed to surface.
- Fairness Testing
- Related: Bias, Disparate Impact, Continuous Monitoring
Structured evaluation of model outcomes and error rates across groups using defined metrics and thresholds, performed before deployment and periodically afterward, with results documented and reviewed by accountable owners.
- Model Risk
- Related: Model Validation, Residual Risk, Explainability
The potential for adverse consequences from decisions or estimates based on incorrect or misused model outputs, including financial loss, reputational harm, regulatory action, and customer detriment.
- Model Validation
- Related: Model Risk, Internal Audit Assurance, Benchmark Evaluation
Independent assessment of whether a model is conceptually sound, performs as intended, and is appropriate for its use, covering data quality, methodology, testing results, and limitations before approval and at defined intervals.
- Residual Risk
- Related: AI Risk Appetite, Accountable Executive, Risk Tolerance Threshold
The risk that remains after controls and mitigations are applied, which must be compared against the organization's risk appetite and formally accepted by an accountable owner before deployment proceeds.
- Risk Tolerance Threshold
- Related: AI Risk Appetite, Residual Risk, Control Testing
A pre-set boundary that translates risk appetite into concrete decision rules, such as error-rate limits or required approval tiers, so that assessors can consistently judge whether a use case may proceed.
- Human in the Loop
- Related: Automation Bias, Guardrail, Hallucination
A design in which a person reviews, approves, or can override AI output before it takes effect, with effectiveness depending on reviewer competence, adequate information, sufficient time, and real authority to reverse the decision.
- Automation Bias
- Related: Human in the Loop, Model Risk, Control Testing
The human tendency to accept automated recommendations uncritically, which weakens the protection a human review step is meant to provide and is countered through training, sampling audits, and disagreement incentives.
- Red Teaming
- Related: Guardrail, Prompt Injection, Control Testing
Adversarial testing in which skilled testers deliberately attempt to make an AI system produce harmful, incorrect, or policy-violating output, documenting successful attack paths so that safeguards can be strengthened before release.
- Control Testing
- Related: Internal Audit Assurance, Risk Tolerance Threshold, Continuous Monitoring
Examination of whether a stated control is designed appropriately and operates effectively in practice, using evidence such as logs, samples, and re-performance, with results feeding remediation and assurance reporting.
- Continuous Monitoring
- Related: Model Drift, Control Testing, Guardrail
Ongoing automated and manual observation of AI system behavior, data inputs, and control performance after deployment, with defined metrics, alert thresholds, and escalation paths so that degradation is detected and acted upon.