ISTQB CT-GenAI cheat sheet

Generative AI (GenAI) produces new content (text, code, images, test data) by learning statistical patterns from training data; it does not retrieve a stored correct answer, so outputs are probabilistic, not deterministic.

Core model families relevant to testing: LLMs for text, code and test cases; diffusion models for images; multimodal models accept mixed inputs (text plus image) and can describe or reason about screenshots.

A token is the model's unit of input/output; context window = maximum tokens (prompt plus response) the model can process at once. Exceeding it truncates or errors, so long requirement documents must be chunked.

Temperature and top-p control sampling randomness: low temperature (near 0) gives repeatable, conservative output suited to regression-style generation; higher values increase diversity for exploratory test ideas.

Hallucination = fluent but factually wrong output (invented APIs, selectors, requirement IDs). Mitigation: ground prompts with source material, require citations to provided text, and verify outputs against the system under test.

Training cut-off and data provenance limit reliability: models may not know recent framework versions or proprietary domain rules, so supply that context explicitly in the prompt.

Prompt engineering patterns: role, task, context, constraints, output format, examples (few-shot). Chain-of-thought or step decomposition improves multi-step reasoning such as deriving boundary values.

Risks and governance: data leakage via prompts, bias inherited from training data, non-determinism hindering reproducibility, IP/licensing of generated code, and the need for human review of all AI output.

GenAI supports test analysis by extracting test conditions from requirements, user stories and acceptance criteria; it drafts candidate conditions that a human tester validates for completeness and traceability.

Equivalence partitioning and boundary value analysis can be drafted by prompting for input domains, valid/invalid classes and boundary triples (min-1, min, max, max+1); the model's arithmetic must be re-checked.

Decision tables and state transition models: the model can enumerate condition combinations and transitions, but coverage of unreachable or contradictory rules requires human pruning.

Test case generation quality depends on the prompt's specification detail; supplying explicit format (ID, preconditions, steps, expected result, priority) yields directly usable cases, otherwise output varies.

Risk-based prioritization: GenAI can propose likelihood/impact ratings from requirement text, but ratings must be calibrated by the team; the model has no knowledge of production incident history unless provided.

Negative and non-functional test ideas (security, accessibility, performance, usability) are a strength of GenAI brainstorming; prompts naming the quality characteristic produce more targeted ideas than open-ended requests.

Ambiguity detection: asking the model to list unclear, untestable or missing requirements surfaces gaps early; treat its findings as review prompts, not verified defects.

Traceability and coverage: AI-drafted cases must be mapped back to requirements and reviewed for duplication and gaps; coverage metrics are computed by the team, not asserted by the model.

GenAI can draft automation code (unit, API, UI tests) and page objects from a described scenario; generated code must be reviewed, compiled and run before acceptance, since APIs and locators may be invented.

Synthetic test data generation: models produce valid, invalid, boundary and edge-case data sets; for privacy-sensitive fields use generated values rather than production data, and validate format constraints programmatically.

Locator and selector suggestions (CSS/XPath) are fragile if based on guessed DOM; the model must be given the actual DOM snippet, and resilient locator strategies should be preferred.

Self-healing test automation uses AI to propose updated locators after UI changes; proposals require validation against the real application to avoid masking genuine defects.

Test oracle support: GenAI can generate expected outputs and assertions, but expected results derived from a possibly wrong model are a risk; cross-check against specifications or independent computation.

Defect triage and analysis: models summarize failures, cluster stack traces and suggest root causes; classification accuracy must be verified by engineers before routing.

Execution-time uses: generating test scripts from natural language, converting manual steps to code, and explaining failing test output; all outputs are drafts subject to the test pyramid and review gates.

Prompt-to-code workflow control: version prompts and generated artifacts, keep human approval in the pipeline, and never let unreviewed AI code merge automatically into main.

GenAI can summarize test execution results, defect reports and logs into stakeholder-facing status reports; source data must be supplied in the prompt because the model cannot see the test management tool.

Report accuracy rule: every figure (pass rate, defect counts, coverage) must be traceable to the underlying data; models can misstate numbers, so totals are recomputed and verified before publication.

Defect report enrichment: models draft clear titles, steps to reproduce and severity suggestions from logs and screenshots, improving triage speed; severity and priority remain human decisions.

Monitoring and log analysis: GenAI clusters production incidents, drafts anomaly explanations and proposes hypotheses; correlation is not causation, so hypotheses require confirmation with telemetry.

Root cause analysis support: prompting with timeline, logs and recent changes yields candidate causes and follow-up questions; the model's output is an input to investigation, not a conclusion.

Continuous improvement: retrospective notes, recurring defect patterns and coverage gaps can be summarized into process actions; metrics definitions must be fixed in advance so AI summaries stay comparable over time.

Feedback loop: false positives, rejected AI suggestions and review corrections are captured as prompt and process improvements, making GenAI use measurable rather than ad hoc.

Governance for reporting: label AI-generated content, retain human sign-off, protect confidential data in prompts, and keep an audit trail of prompts and model versions used for regulated reports.