ISTQB CT-AI study plan

Distinguish AI, machine learning, deep learning, and generative AI with a one-line definition and one example of each

Sequence the stages of the ML workflow from problem framing through data preparation, training, evaluation, deployment, and monitoring

Classify a given ML use case as supervised, unsupervised, or reinforcement learning and justify the classification

Map at least six testing activities (data checks, model evaluation, integration tests, monitoring checks) onto the lifecycle stages where they belong

Derive test conditions for training, validation, and test datasets from the data quality characteristics of completeness, correctness, consistency, balance, and representativeness

Design equivalence-partition and boundary-value checks applicable to numeric and categorical input features

Identify eight distinct data defects (label noise, duplicates, leakage, missing values, outliers, sampling bias, schema drift, imbalance) and state how each would be detected

Plan tests for a data preparation pipeline including ingestion, transformation, splitting, and labelling steps

Explain how train/test leakage and non-representative splits distort evaluation results

Compute and interpret a confusion matrix, accuracy, precision, recall, specificity, and F1 score by hand for a small worked example

Select an appropriate evaluation metric given a stated business cost of false positives versus false negatives

Diagnose overfitting and underfitting from training-versus-validation learning curves and propose a remedy for each

Contrast model testing approaches: held-out test sets, k-fold cross-validation, and metamorphic/adversarial input testing

Distinguish functional performance testing of a model from testing of non-functional characteristics such as robustness and efficiency

Describe what is tested at each ML development level: data pipeline code, feature engineering code, training code, model artifact, and model integration into a system

Explain how CI/CD practices are adapted for ML, including model versioning, data versioning, and automated retraining triggers

Identify unit, integration, system, and acceptance test objectives that apply specifically to ML components

Define review and static analysis activities appropriate for notebooks, configuration files, and hyperparameter settings

Distinguish testing of the ML pipeline from testing of the surrounding application

Enumerate LLM-specific risk areas: hallucination, prompt injection, data leakage in prompts, bias, toxicity, non-determinism, and cost/latency variability

Write prompt-based test cases that state input prompt, expected behaviour, and acceptance criteria for a defined task

Compare evaluation strategies for generative output: human review, rubric scoring, automated similarity or model-based scoring, and benchmark suites

Explain how retrieval-augmented generation changes the test surface, including retrieval accuracy and grounding checks

Assess the limits of exact-match oracles when the same input can produce different acceptable outputs

List the AI-specific quality characteristics and describe what each means for testing an AI-based system

Relate functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability, and portability to AI-specific concerns such as autonomy, adaptability, and transparency

Explain how the AI quality model extends a conventional software product quality model rather than replacing it

Design test objectives for two characteristics chosen from the model, naming observable evidence that would show each is met

Identify which characteristics are hardest to verify objectively and justify the difficulty

Re-test the three weakest topics identified from earlier self-assessment using fresh questions

Complete a timed full-length practice set and classify every incorrect answer by domain and by error type (knowledge gap, misreading, guessing)

Re-derive from memory the metrics formulas and the data quality characteristics that were previously missed

State in one sentence, for each of the seven domains, the two facts most likely to be examined

Consolidate definitions and lifecycle terminology across the foundational domains using spaced-repetition recall

Complete a second timed practice set and compare the domain-by-domain result with week 7

Rewrite each personal error-log entry as a one-line rule that can be recalled under time pressure

Verify that all seven domains have been covered at least twice across the plan