ISTQB CT-AI cheat sheet

An AI-based system is a software or hardware system that, for a defined set of objectives, produces outputs such as predictions, recommendations or decisions using data-driven or knowledge-based reasoning.

Symbolic AI (rule-based systems, expert systems, ontologies) reasons from explicitly encoded knowledge; ML-based AI induces behaviour from data; hybrid systems combine both approaches.

Three core ML paradigms: supervised learning (labelled data), unsupervised learning (structure in unlabelled data) and reinforcement learning (reward-driven interaction); semi-supervised and self-supervised sit between them.

Narrow AI performs a bounded task (classification, recommendation, detection); general AI would match humans across arbitrary domains and does not exist in deployed systems today.

Deep learning uses artificial neural networks with many layers whose parameters are learned by gradient descent with backpropagation.

AI-specific testing challenges: non-determinism, probabilistic outputs, the oracle problem, limited explainability, dependence on training data and behaviour that changes after learning.

Data-dependent behaviour: the same code trained on different data is effectively a different system under test, so data must be versioned and tested as part of the system.

The 'AI effect': once a capability becomes routine it stops being called AI, so the boundary and scope of AI testing shift over time.

ISO/IEC 25059 extends the ISO/IEC 25010 SQuaRE product quality model to AI systems, retaining the familiar characteristics and adding or adapting AI-specific ones.

Top-level characteristics include functional suitability, performance efficiency, compatibility, interaction capability, reliability, security, maintainability, flexibility and safety.

Portability of ISO/IEC 25010 is replaced by flexibility, covering adaptability to new data, environments, tasks and contexts of use, plus scalability and replaceability.

Safety is added as a characteristic: freedom from unacceptable risk of harm to people, property or the environment across the lifecycle, including foreseeable misuse.

Functional suitability (functional completeness, correctness, appropriateness) must be specified against measurable acceptance thresholds rather than 'the model works'.

Reliability includes robustness: acceptable performance must hold under noise, distribution shift, adversarial input and degraded or missing data.

Security must address data poisoning, adversarial/evasion input, model extraction, membership inference and insecure handling of model outputs.

Interaction capability covers users' understanding of AI limits (confidence, uncertainty, known failure modes) and the ability to override or correct system output.

Quality in use (effectiveness, efficiency, satisfaction, freedom from risk, context coverage) is measured with real users in the operational context, not on benchmark data alone.

CRISP-DM phases: business understanding, data understanding, data preparation, modelling, evaluation and deployment; the process is iterative and can loop back from any phase.

Parameters are learned from data during training; hyperparameters are set before training and tuned using the validation set, never the test set.

Split strategies: hold-out for large data, k-fold cross-validation for small data, stratified folds for imbalanced classes, time-ordered splits for temporal data, group splits to avoid per-subject leakage.

Scaling and encoding statistics must be computed on the training fold only and then applied to validation/test data; otherwise data leakage inflates reported quality.

Bias-variance: underfitting shows high error on training and test data (high bias); overfitting shows low training error and high test error (high variance); regularization, more data or a simpler model reduce variance.

MLOps: version control for data, code, model and configuration; CI/CD plus continuous training (CT); model registry with lineage from dataset to deployed artefact.

Monitoring covers data drift (covariate shift), label drift, concept drift and delayed ground truth availability; retraining is triggered by defined thresholds.

Deployment patterns include shadow, canary, A/B and champion/challenger, each with pre-defined success criteria and rollback conditions.

Data quality dimensions to test: completeness, accuracy, consistency, timeliness, uniqueness, validity and balance/representativeness.

Test data must represent the operational distribution and must explicitly include edge cases, corner cases, rare classes and known failure-prone segments.

Label quality: measure inter-annotator agreement (e.g., Cohen's kappa), adjudicate disagreements and audit a sample of labels per class before training.

Leakage checks include duplicate or near-duplicate rows across splits, target-derived features, temporal leakage and group leakage (same subject in train and test).

Bias checks: class imbalance, sampling bias and proxy features for protected attributes; fairness metrics computed per subgroup (e.g., demographic parity, equalized odds).

Privacy checks: detect PII/PHI, pseudonymise or synthesise data, and confirm purpose limitation and retention rules before using production data for testing.

Synthetic and augmented data must be validated for fidelity and for not masking real-world gaps; adversarial and outlier data are injected to test robustness.

Data versioning and lineage: every model must be traceable to the exact dataset snapshot and transformation pipeline used to produce it.

Test levels: data and code unit tests, model-level tests of the trained model, system/integration tests of the model in the pipeline, and acceptance tests against business requirements.

Classification metrics from the confusion matrix: accuracy, precision, recall, specificity, F1, ROC-AUC and PR-AUC; regression metrics: MAE, MSE, RMSE and R-squared.

Metrics must be reported per class and per slice (subgroup, segment, time window); a good average can conceal a failing subgroup.

Oracle problem workarounds: metamorphic relations, invariants, differential/back-to-back comparison, tolerance-based statistical oracles and structured human evaluation.

Non-determinism handling: fix seeds, repeat runs and accept a result only if the metric distribution meets the defined threshold.

Robustness tests: noise, blur and perturbation, adversarial examples (e.g., FGSM), missing features and out-of-distribution inputs.

Fairness testing compares error rates and outcomes across subgroups; a model that is fair per group can still be unfair in the full decision context.

Performance and resource testing: inference latency, throughput, memory and cost per prediction under expected and peak load.

Structural coverage for neural networks includes neuron coverage, threshold coverage and behaviour coverage; these are necessary but not sufficient for test adequacy.

Development testing covers unit tests for data ingestion, feature transformations, training code and the serving API, not only the model itself.

Reproducibility checks: fixed random seeds, pinned dependencies, versioned data and configuration, deterministic ordering; another engineer must be able to repeat the run.

Data tests inside the pipeline: schema, value ranges, null rates and distribution comparison between training data and incoming data (expectation-based checks).

Model validation uses cross-validation and hyperparameter search isolated from the test set, with results recorded in the model registry.

Regression suites: golden datasets with expected tolerances, model-level regression tests on every retrain, and comparison against the current champion model.

Integration testing must detect training-serving skew by comparing feature values computed in the pipeline with those computed at inference time.

Deployment testing: shadow, canary or A/B runs with defined success criteria and automatic rollback; monitoring and alerting verified before go-live.

Documentation review: model cards and datasheets covering intended use, training data, metrics, limitations and known risks must be consistent with test evidence.

Outputs are non-deterministic due to sampling and temperature, so 'same input, same output' is not a valid oracle; use rubrics, thresholds and statistical acceptance instead.

Evaluation methods: reference-based metrics (BLEU, ROUGE, BERTScore, exact match), human rating scales, and LLM-as-a-judge only after validating judge agreement and bias.

RAG evaluation metrics: context relevance, context precision and recall, faithfulness/groundedness, and answer relevance.

Hallucination testing: ask questions with verifiable answers, check grounding and citation against sources, and include out-of-scope and unanswerable prompts.

Security testing: direct and indirect prompt injection (including via retrieved content), jailbreaks, system-prompt leakage, PII exfiltration and insecure output handling.

Safety and red teaming: toxicity, bias, harmful or illegal instructions and guardrail bypass attempts, tested both before and after mitigation.

Robustness and consistency: paraphrase, typos, formatting changes, multilingual prompts, long contexts and repeated runs, checking answer stability.

Fine-tuning evaluation must include regression tests on capabilities the base model already had, to detect catastrophic forgetting.

Cost and performance: token usage, latency, rate limits and behaviour at context-window boundaries.