ISTQB CT-AI key terms
ISTQB CT-AI · Get comfortable with the vocabulary.
- Artificial intelligence (AI)
- Related: Machine learning (ML), AI-based system
The branch of computing that builds systems able to perform tasks usually needing human intelligence, such as recognising images, understanding language, or choosing actions.
- AI-based system
- Related: Machine learning (ML), Trustworthiness
A software system in which some behaviour or output is produced by AI components, such as trained models, rather than only by explicitly written program logic.
- Machine learning (ML)
- Related: Training, Deep learning
An approach in which algorithms learn patterns from data and adjust themselves to build a model, instead of following rules that a person writes by hand.
- Deep learning
- Related: Machine learning (ML), Overfitting
A machine learning technique that stacks many layers of simple processing units so the system can learn complex patterns from large volumes of data.
- Expert system
- Related: Machine learning (ML), AI-based system
A rule-driven system that stores human expertise in a knowledge base and applies it to give advice in a narrow domain, without learning from examples.
- ISO/IEC 25059
- Related: Trustworthiness, Functional correctness
An international standard that extends the SQuaRE quality model with characteristics and guidance specific to AI-based systems, helping teams specify and evaluate their quality.
- Functional correctness
- Related: Test oracle, Safety
The degree to which an AI-based system produces the outputs that are right for the task, measured against agreed references or ground truth.
- Robustness
- Related: Adversarial example, Safety
The ability of a model or system to keep giving acceptable results when inputs are noisy, unusual, corrupted, or deliberately crafted to cause failure.
- Transparency
- Related: Explainability, Trustworthiness
The degree to which information about an AI system, such as its purpose, data, logic, and limits, is available so stakeholders can understand and question it.
- Explainability
- Related: Transparency, Trustworthiness
The ability to give people meaningful reasons for an AI system's output, often using additional techniques that approximate how the model reached its answer.
- Fairness
- Related: Data bias, Trustworthiness
The property that an AI system treats individuals and groups comparably, without unjustified differences in outcomes tied to protected characteristics such as gender or ethnicity.
- Trustworthiness
- Related: Robustness, Fairness, Transparency
The overall property of an AI system being safe, fair, reliable, transparent, and secure enough that people can reasonably depend on it.
- Safety
- Related: Robustness, Trustworthiness
The property that an AI-based system does not cause unacceptable harm to people, property, or the environment, including when it behaves in unexpected ways.
- Machine learning lifecycle
- Related: Training, Concept drift
The ordered activities from defining a problem through data work, model building, deployment, monitoring, and retirement that create and maintain an ML system.
- Training
- Related: Inference, Overfitting
The process of running a learning algorithm over data so it adjusts a model's internal parameters and improves its ability to perform the intended task.
- Inference
- Related: Training, Machine learning lifecycle
Using an already trained model to produce predictions or content for new inputs, without changing the model's learned parameters.
- Label
- Related: Training data, Test oracle
The known or reference answer attached to a data item, used to teach a supervised model and to judge whether its predictions are right.
- Hyperparameter
- Related: Training, Overfitting
A setting chosen before training that controls how a learning algorithm works, such as learning rate or tree depth, and is not learned from the data.
- Overfitting
- Related: Generalisation, Training
A failure where a model learns its training data too closely, including its noise, so it scores well there but badly on new data.
- Concept drift
- Related: Training data, Machine learning lifecycle
A change over time in the true relationship between inputs and correct outputs, so a model that was accurate gradually becomes less accurate.
- Generalisation
- Related: Overfitting, Test data
A model's ability to give good results on data it has never seen, which is the real purpose of machine learning.
- Training data
- Related: Test data, Data quality
The examples used to fit a model's parameters during learning; their quality, coverage, and balance strongly shape what the model learns.
- Test data
- Related: Training data, Data leakage
Data kept apart from training and used only to measure how well a finished model performs on cases it has not encountered.
- Data quality
- Related: Data bias, Training data
The extent to which data is accurate, complete, consistent, timely, and relevant enough for the purpose it will serve.
- Data bias
- Related: Fairness, Training data
A systematic distortion in a dataset that pushes a model toward skewed or unfair patterns, for example through unrepresentative sampling or labelling habits.
- Data leakage
- Related: Test data, Training data
A mistake where information from the target or from test data reaches training, giving results that look excellent but fail on real new data.
- Test oracle
- Related: Label, Metamorphic testing
A source of expected results used to judge whether an observed output is correct; for ML this is often labelled data or a derived property.
- Metamorphic testing
- Related: Test oracle
A technique for checking a model without expected outputs, by confirming that related inputs produce outputs related in a predicted way.
- Precision
- Related: Recall
Of all the items a model labelled positive, the share that really are positive; high precision means few false alarms.
- Recall
- Related: Precision
Of all the items that really are positive, the share the model found; high recall means few positive cases were missed.
- Adversarial example
- Related: Robustness
An input deliberately modified in a way people barely notice but which causes a model to give a wrong or unsafe result.
- ML development testing
- Related: Pipeline testing, Training data
Testing the code, data handling, and components around a model while it is being built, as distinct from measuring the model's predictive quality.
- Pipeline testing
- Related: ML development testing, Data quality
Checking that the chain of data preparation, training, and serving steps works end to end and handles bad data or changed formats correctly.
- Shadow deployment
- Related: ML development testing, Concept drift
Running a new model beside the current one on live inputs, recording its outputs without acting on them, so behaviour can be compared safely.
- Generative AI
- Related: Large language model (LLM), Prompt
AI that produces new content such as text, images, audio, or code by learning patterns from large collections of existing examples.
- Large language model (LLM)
- Related: Generative AI, Hallucination
A very large neural network trained on huge amounts of text to predict and generate language, used for answering, summarising, translating, and similar tasks.
- Prompt
- Related: Large language model (LLM), Hallucination
The instructions or input text given to a generative model, which strongly influence the response that comes back.
- Hallucination
- Related: Large language model (LLM), Prompt
When a generative model states something false or invented with confidence, instead of admitting that it does not know.
- Prompt injection
- Related: Guardrails, Prompt
An attack in which hidden instructions in input or retrieved content cause a model to ignore its intended instructions or reveal protected information.
- Guardrails
- Related: Prompt injection, Hallucination
Checks and controls around a generative model, such as filters, content rules, and output limits, that keep responses safe and within scope.