ISTQB CT-AI study guide
ISTQB CT-AI · Work through your exam, one topic at a time.
ISTQB® Certified Tester AI (CT-AI) is a specialist certification for professionals who test AI-based systems. Version 2.0 restructured the syllabus from 11 chapters down to 7 and reorganized it around the machine learning lifecycle - input data testing, model testing and ML development testing - while narrowing the scope exclusively to testing AI-based systems. The ISTQB CT-AI study guide has 7 sections and about 2,331 words.
01Introduction to AI and AI-based Systems
Artificial intelligence is an umbrella term. Knowledge-based (symbolic) AI encodes human expertise as explicit rules - expert systems, rule engines, decision tables.
343 words, about 2 min02Quality Characteristics of AI-based Systems (ISO/IEC 25059)
ISO/IEC 25059 defines a quality model for AI systems by extending ISO/IEC 25010 (product quality). The familiar characteristics remain - functional suitability, performance efficiency, compatibility, interaction capability, reliability, security, maintainability, flexibility - but they are re-interpreted for learned behaviour, and AI-specific sub-characteristics are added or emphasised.
340 words, about 2 min03Machine Learning Lifecycle and ML Workflows
A widely used reference is CRISP-DM : business understanding → data understanding → data preparation → modelling → evaluation → deployment, and it is explicitly iterative , not a waterfall. Modern practice wraps this in MLOps: monitoring in production, drift detection, retraining, and versioning of data, code and models.
338 words, about 2 min04Input Data Testing
In ML the training data effectively is part of the specification, so data defects become product defects. Test data quality across dimensions such as completeness, accuracy, consistency, validity, uniqueness, timeliness and balance .
311 words, about 2 min05Model Testing
Start from the confusion matrix (TP, FP, FN, TN). Accuracy alone is misleading under imbalance: a 99:1 dataset yields 99% accuracy by always predicting the majority class (the accuracy paradox ).
316 words, about 2 min06ML Development Testing
ML development testing covers (a) the conventional code - data pipelines, feature computation, training scripts, serving APIs - and (b) model behaviour . Unit and integration testing apply to (a) in the usual way; (b) needs metric-based and statistical approaches.
342 words, about 2 min07Testing Generative AI and Large Language Models
Outputs are open-ended text, so there is no single correct answer. Behaviour is probabilistic (sampling temperature/top-p) and prompt-sensitive : small wording changes can alter output.
341 words, about 2 min