I85PREP
⌘ K
ISTQB CT-GenAI study materials
ISTQB CT-GenAI study resources

ISTQB CT-GenAI practice questions

One question at a time. Build your ISTQB CT-GenAI understanding.

First 50 of 200 questions. Each one lists the choices, the correct answer, and an explanation. The checker at the top of this page saves progress in this browser when it is available.

Continue in the practice tool above to load more questions, 50 at a time.

Question 1Fundamentals of Generative AI for Testing

In the ISTQB CT-GenAI vocabulary, what distinguishes a large language model (LLM) from a classical rule-based test data generator?

  1. The LLM stores a lookup table of every input it has ever seen and replays the closest match
  2. The LLM produces deterministic output for identical prompts because sampling is disabled by default
  3. The LLM predicts probable token sequences learned from training data rather than executing hand-coded generation rules Correct answer.
  4. The LLM guarantees syntactically valid output because its decoder enforces a formal grammar at every step

Explanation

A generative model such as an LLM is trained to estimate a probability distribution over token sequences and then samples or decodes from it, so its behaviour is statistical rather than rule-driven. Classical generators follow explicit programmer-defined logic, which is why their outputs are fully predictable and traceable to source code. This statistical nature is exactly why generative AI output in testing must be validated rather than trusted blindly.

Question 2Fundamentals of Generative AI for Testing

A test team wants the generative AI tool to propose boundary values for an age field that accepts 18 to 65. Which prompt element most directly improves the relevance of the generated test data?

  1. A polite closing sentence thanking the model
  2. A request for a long creative narrative about the field
  3. An explicit statement of the valid range and the boundary convention Correct answer.
  4. A request to answer in rhyming verse

Explanation

Generative AI output quality depends heavily on the context supplied in the prompt. Stating the accepted range, the inclusive or exclusive convention, and the expected output format gives the model the constraints it needs to produce relevant boundary values. Without such constraints the model may invent plausible but incorrect limits.

Question 3Fundamentals of Generative AI for Testing

Which statement best describes a hallucination in the context of generative AI used for software testing?

  1. A deliberate refusal by the model to answer a prompt
  2. A formatting error introduced by the test harness
  3. A fluent output that asserts facts or artefacts the model has no basis for Correct answer.
  4. A network timeout that interrupts the model response

Explanation

Hallucination refers to content that is generated fluently and confidently but is not grounded in the provided input or in verifiable reality, such as an invented API method or a non-existent requirement identifier. For testing, this means generated test cases and expected results must be checked against the actual system under test. Hallucination is a property of model output, not of infrastructure failures.

Question 4Fundamentals of Generative AI for Testing

What is the primary purpose of a retrieval-augmented generation (RAG) architecture in a test engineering context?

  1. To ground model responses in retrieved authoritative sources such as the test basis Correct answer.
  2. To fine-tune the model weights on every new requirement document
  3. To reduce the model parameter count so it runs on a laptop
  4. To replace the test oracle with the model's own judgement

Explanation

RAG retrieves relevant passages from an external knowledge base, such as requirements, specifications, or defect history, and supplies them to the model as context at inference time. This grounding reduces fabrication and lets answers cite the actual test basis without retraining. It does not change the model weights or remove the need for human review.

Question 5Fundamentals of Generative AI for Testing

A team uses a generative AI assistant to draft test cases from a user story. Which activity belongs to human review rather than to the model?

  1. Producing candidate test steps from the story text
  2. Reformatting the steps into a table
  3. Confirming that each expected result matches the agreed acceptance criteria Correct answer.
  4. Suggesting additional negative scenarios

Explanation

The model can generate candidate steps, formats, and ideas, but it cannot know whether an expected result is the correct business outcome. Confirming alignment with acceptance criteria is an oracle decision that requires the test basis and accountable human judgement. This division of labour is central to responsible generative AI use in testing.

Question 6Fundamentals of Generative AI for Testing

Which characteristic most clearly separates generative AI from conventional classification models used in test tooling?

  1. Classifiers can only process images, never text
  2. Classifiers always run locally, while generative models never do
  3. Generative models require no training data at all
  4. Generative models output newly synthesised content, while classifiers assign a label to given input Correct answer.

Explanation

A classifier maps an input to one of a predefined set of labels, for example predicting whether a log entry indicates a defect. A generative model instead produces new sequences, such as test steps, data, or summaries. Both rely on training data, so the presence or absence of training data is not the distinguishing factor.

Question 7Fundamentals of Generative AI for Testing

What does the temperature parameter control when a generative AI model produces test ideas?

  1. The randomness of sampling, trading diversity against consistency Correct answer.
  2. The number of retrieved documents added to the prompt
  3. The amount of training data used for the current request
  4. The maximum number of tokens the model may output

Explanation

Temperature scales the probability distribution used during sampling. Lower values make the model favour high-probability tokens and produce more consistent, repeatable output, while higher values increase variety at the cost of predictability. It does not limit length, retrieval volume, or training scope.

Question 8Fundamentals of Generative AI for Testing

Why is it risky to accept generated test data containing realistic-looking personal information without checks?

  1. Personal data improves the determinism of the test suite
  2. Personal data may be fabricated yet resemble real individuals, creating privacy and compliance exposure Correct answer.
  3. Generated data is automatically anonymised by the model
  4. Generated data is always encrypted, so it cannot be inspected

Explanation

Generative models can produce plausible names, addresses, and identifiers that may accidentally match or resemble real people, and prompts may also leak personal data into logs or third-party services. Test data management therefore requires review for privacy and compliance before use. Encryption and automatic anonymisation are not guaranteed properties of generation.

Question 9Fundamentals of Generative AI for Testing

A model reliably generates valid-looking JSON test fixtures, but the field names do not match the system's API schema. Which prompt improvement addresses this most directly?

  1. Reduce the output length to a single line
  2. Increase the temperature to encourage variety
  3. Include the exact schema or a representative example in the prompt Correct answer.
  4. Ask the model to explain its reasoning in prose

Explanation

The mismatch is a grounding problem, not a creativity problem. Supplying the authoritative schema, or a concrete example of a valid fixture, constrains the model to the real field names and types. Raising temperature would make schema drift more likely, and prose reasoning does not supply the missing schema.

Question 10Fundamentals of Generative AI for Testing

Which scenario is the most appropriate use of generative AI within a test process?

  1. Replacing the requirement review meeting entirely
  2. Deleting failing automated tests that the model judges obsolete
  3. Drafting exploratory test charters and candidate scenarios for tester review Correct answer.
  4. Approving a release based on the model's confidence score

Explanation

Generative AI is well suited to producing drafts and ideas that humans then evaluate, such as exploratory charters and scenario candidates. Decisions with accountability, such as release approval, test deletion, or waiving stakeholder review, require human ownership and traceability. Confidence scores are model-internal signals and are not a valid release gate.

Question 11Fundamentals of Generative AI for Testing

A team notices that their generative assistant produces excellent test cases for the payment domain but poor ones for an internal legacy protocol. Which explanation is most consistent with how LLMs work?

  1. The model deliberately withholds knowledge to force human review
  2. The legacy protocol uses characters the tokenizer cannot represent
  3. The model's training data contained little material about the niche legacy protocol Correct answer.
  4. The assistant's network latency caused lower-quality answers

Explanation

LLM capability is uneven across domains because it reflects the distribution of the training data. Well-documented areas such as payments appear frequently, whereas a proprietary legacy protocol may be barely represented, leading to weaker or fabricated output. The remedy is to supply context through prompting or retrieval and to review results carefully.

Question 12Fundamentals of Generative AI for Testing

What is the main testing benefit of using a generative model to summarise a long defect log before triage?

  1. It guarantees the summary contains every defect
  2. It removes the need to verify the underlying defects
  3. It permanently replaces the defect tracking system
  4. It condenses large volumes of text so testers can prioritise faster Correct answer.

Explanation

Summarisation reduces reading effort by condensing repetitive text, letting testers focus attention on prioritisation and analysis. The summary is a lossy transformation, so details may be omitted or distorted and the source records remain authoritative. It therefore supports rather than replaces verification and tracking.

Question 13Fundamentals of Generative AI for Testing

A prompt asks a model to 'write tests for the login page'. The output is generic and misses the organisation's multi-factor authentication rules. Which refinement is most effective?

  1. Ask the model to be more creative about security
  2. Add the relevant authentication rules, user roles, and expected output format to the prompt Correct answer.
  3. Repeat the same prompt several times and pick the longest answer
  4. Request the answer in a different natural language

Explanation

Generic output usually signals missing context. Supplying the specific business rules, the roles involved, and the desired structure gives the model the constraints needed to generate relevant tests. Repeating an under-specified prompt or changing style or language does not inject the missing domain knowledge.

Question 14Fundamentals of Generative AI for Testing

When a generative AI tool is used to create synthetic test data for a database, which control best supports repeatability of test runs?

  1. Regenerating data on every execution to maximise variety
  2. Disabling logging of prompts to save storage
  3. Relying on the model's default sampling each time data is needed
  4. Persisting the generated dataset and its prompt so runs reuse the same records Correct answer.

Explanation

Repeatability requires that the inputs to a test remain stable across runs. Storing the generated dataset together with the prompt that produced it turns a non-deterministic generation step into a fixed, auditable artefact. Regenerating each time introduces variation that can make failures hard to reproduce.

Question 15Fundamentals of Generative AI for Testing

A tester uses a generative assistant to propose expected results for a discount calculation. What is the correct treatment of those proposals?

  1. Verify them against the pricing specification or an independent calculation Correct answer.
  2. Use them as the sole oracle for the regression suite
  3. Accept them if the explanation sounds logically consistent
  4. Accept them only when the model expresses high confidence

Explanation

Expected results are oracle information and must be traceable to an authoritative source such as a specification or an independent computation. Plausible reasoning or self-reported confidence is not evidence of correctness, because models can produce coherent but wrong derivations. Generated expectations therefore serve as candidates for verification, not as the oracle itself.

Question 16Fundamentals of Generative AI for Testing

Which risk is specific to using a hosted third-party generative AI service during test design?

  1. Automated tests may execute in the wrong order
  2. The model may produce output faster than testers can read it
  3. The test management tool may run out of disk space
  4. Confidential requirement text sent in prompts may leave the organisation's control Correct answer.

Explanation

Sending internal requirements, code, or defect details to an external service can disclose confidential information and may conflict with data protection obligations or contractual terms. This is a governance risk specific to third-party generative services. Disk space, reading speed, and execution order are ordinary project concerns unrelated to the model.

Question 17Fundamentals of Generative AI for Testing

What does fine-tuning a pre-trained model for a testing task actually change?

  1. It rewrites the tokenizer to remove rare words
  2. It updates model parameters using task-specific examples so behaviour adapts to that domain Correct answer.
  3. It adds a retrieval index of documents consulted at inference time
  4. It guarantees the model will no longer hallucinate

Explanation

Fine-tuning continues training on a curated dataset so the model's weights shift toward the target task, style, or domain. It is distinct from retrieval, which supplies external context at inference time without changing weights. Fine-tuning can reduce some errors but does not eliminate hallucination, because the model remains probabilistic.

Question 18Fundamentals of Generative AI for Testing

A team wants generated test cases to be traceable to specific requirement clauses. Which combination best supports that goal?

  1. Higher temperature plus longer outputs
  2. Free-form prose output plus manual retyping
  3. A smaller model plus a shorter prompt
  4. Retrieval of requirement text plus a prompt requiring clause identifiers in the output Correct answer.

Explanation

Traceability needs both the authoritative source and an output contract. Retrieval supplies the actual clauses, and the prompt instructs the model to cite their identifiers in a defined structure. Neither higher temperature nor shorter prompts provide the source material or the citation requirement.

Question 19Fundamentals of Generative AI for Testing

Which statement correctly describes the relationship between tokens and model context limits?

  1. Tokens are characters, and the context limit counts only the model's output
  2. Tokens are whole sentences, so context limits rarely matter for test cases
  3. Tokens apply only to programming languages, not natural language
  4. Tokens are subword units, and the context window bounds input plus output tokens together Correct answer.

Explanation

Text is split into tokens, which are often subword units rather than characters or sentences. The context window is a shared budget covering the prompt, any retrieved content, and the generated response, so long requirement documents can crowd out output space. Managing context therefore matters when feeding large test bases to a model.

Question 20Fundamentals of Generative AI for Testing

An organisation requires that every AI-generated test artefact be attributable and reviewable. Which practice best satisfies this requirement?

  1. Allowing testers to accept output without documenting its origin
  2. Deleting the prompt history after generation to save space
  3. Recording the prompt, model version, and reviewer sign-off alongside the artefact Correct answer.
  4. Relying on the model vendor to retain request logs

Explanation

Attribution and reviewability depend on provenance records linking an artefact to the prompt, the model version, and the human who approved it. This supports audits, reproducibility, and accountability. Discarding prompts or delegating retention to the vendor breaks the organisation's own evidence chain.

Question 21Fundamentals of Generative AI for Testing

Why can the same prompt produce different test cases on two consecutive executions of a generative model?

  1. The tokenizer randomly discards words
  2. The context window shrinks after each call
  3. The model retrains itself between requests
  4. Sampling introduces stochasticity in token selection Correct answer.

Explanation

Decoding samples from a probability distribution, so unless the temperature is zero and the seed is fixed, different tokens may be chosen on each run. The model weights are not updated between ordinary inference requests. Tokenizers are deterministic and context windows do not shrink as a side effect of use.

Question 22Fundamentals of Generative AI for Testing

A team must choose between prompting, retrieval, and fine-tuning to make a model follow their internal test documentation template consistently. Which reasoning is most sound?

  1. Prompting with a template example is the cheapest first step, with retrieval added if reference content is needed Correct answer.
  2. Fine-tuning is mandatory because prompting can never enforce formatting
  3. Retrieval alone guarantees template adherence without any instruction
  4. Fine-tuning should be tried before any prompt is written

Explanation

Format adherence can often be achieved by giving the model a template and an example in the prompt, which is inexpensive and quick to iterate. Retrieval becomes valuable when the task also needs specific reference content, and fine-tuning is a heavier option for persistent style or domain adaptation. Starting with the lightest effective technique is the sound engineering sequence.

Question 23Fundamentals of Generative AI for Testing

A generated test suite passes review, but later the underlying system changes and the suite silently tests obsolete behaviour. Which governance weakness does this expose?

  1. Use of retrieval instead of fine-tuning
  2. Insufficient model temperature during generation
  3. Excessive token count in the original prompt
  4. The absence of a process to re-validate generated artefacts when the test basis changes Correct answer.

Explanation

Generated artefacts age like any other testware: when requirements or the system change, they must be re-checked against the new test basis. Without a maintenance trigger, a once-valid suite can quietly diverge from current behaviour. Temperature, prompt length, and the retrieval-versus-fine-tuning choice do not address lifecycle drift.

Question 24Fundamentals of Generative AI for Testing

Two testers get different generated outputs for the same requirement and both claim theirs is correct. Which evaluation approach best resolves this objectively?

  1. Ask the model to judge which of its own outputs is better
  2. Score both outputs against a predefined rubric derived from the acceptance criteria Correct answer.
  3. Prefer the output that is longer and better formatted
  4. Accept the output from the more detailed prompt

Explanation

Objective comparison requires a stable reference. A rubric built from acceptance criteria defines coverage, correctness, and clarity criteria in advance, so outputs can be scored consistently and disagreements become evidence-based. Prompt detail, length, and formatting are proxies that do not establish correctness.

Question 25Fundamentals of Generative AI for Testing

An LLM-based assistant confidently invents a non-existent method in the system under test and writes tests calling it. Which mitigation addresses the root cause most directly?

  1. Grounding generation in the actual API specification and validating calls against it Correct answer.
  2. Asking the model to add more comments to the generated code
  3. Increasing the output token limit to allow fuller explanations
  4. Lowering the temperature to zero so the model cannot invent anything

Explanation

The root cause is missing grounding in the real interface, so the model fills the gap with plausible invention. Supplying the authoritative API specification through prompt or retrieval, then validating generated calls against that specification, addresses the cause. Temperature and output length change sampling behaviour but cannot supply facts the model lacks.

Question 26Fundamentals of Generative AI for Testing

In ISTQB generative-AI testing terminology, what does the term 'hallucination' describe when a large language model produces content for a test artefact?

  1. Output that is returned with a lower sampling temperature than requested
  2. Output that exceeds the maximum token limit of the model context
  3. Output that is fluent and plausible but factually unfounded or invented Correct answer.
  4. Output that is cached and replayed from a previous identical prompt

Explanation

Hallucination refers to the model generating confident, well-formed content that is not grounded in fact, such as an invented API method or a fabricated requirement clause. Because the text looks plausible, reviewers must verify generated artefacts against authoritative sources rather than judging fluency.

Question 27Fundamentals of Generative AI for Testing

Which description best captures what a 'prompt' is in the context of using generative AI for software testing?

  1. The log file that records which test cases the model has already produced
  2. The labelled dataset used to adjust the model's weights during training
  3. The instruction and any supplied context that conditions the model's generated response Correct answer.
  4. The compiled binary of the model that is executed on the test workstation

Explanation

A prompt is the input text, including instructions, examples and context, that steers the model's output at inference time. It is not a trained artefact or a runtime binary, so prompt quality directly shapes the usefulness of generated test artefacts.

Question 28Fundamentals of Generative AI for Testing

A test manager is documenting the main building blocks of a large language model pipeline used for test generation. Which element is responsible for converting raw text into the numeric units the model processes?

  1. The tokenizer Correct answer.
  2. The optimiser
  3. The retrieval index
  4. The guardrail filter

Explanation

The tokenizer splits input text into tokens and maps them to numeric identifiers that the model can process. Tokens are therefore the unit in which prompts, outputs and context limits are measured.

Question 29Fundamentals of Generative AI for Testing

Which statement best describes the role of a generative AI model within a test automation architecture?

  1. It produces candidate artefacts such as test ideas or scripts that are then validated by deterministic tooling and humans Correct answer.
  2. It replaces the system under test by simulating all business logic during execution
  3. It removes the need for a test oracle because its outputs are self-verifying
  4. It guarantees that generated assertions match the implemented behaviour

Explanation

Generative models are assistive components that propose content; execution, assertions and oracles remain the responsibility of deterministic tooling and human review. Treating generated output as pre-validated would remove the checks that catch plausible but wrong content.

Question 30Fundamentals of Generative AI for Testing

Which factor makes generative AI output non-deterministic even when the same prompt is submitted twice?

  1. Lossless compression applied to the prompt before inference
  2. The immutable weights stored in the model checkpoint
  3. Stochastic sampling from a probability distribution over tokens Correct answer.
  4. The fixed order in which tokens are encoded by the tokenizer

Explanation

Generation selects each next token by sampling from a probability distribution, so runs can diverge unless sampling is constrained, for example by setting temperature to zero where the provider supports it. Encoding and stored weights are deterministic and do not explain variation.

Question 31Fundamentals of Generative AI for Testing

What is 'synthetic test data' as produced by a generative AI tool?

  1. A snapshot of the test environment database restored before each run
  2. Artificially created data that mimics required characteristics without being copied from production records Correct answer.
  3. Test data generated only by random number functions in the test harness
  4. Production data masked after extraction from the live database

Explanation

Synthetic data is fabricated to satisfy specified characteristics such as format, distribution or edge conditions, rather than being taken from real records. This reduces privacy exposure while still exercising the system with realistic inputs.

Question 32Fundamentals of Generative AI for Testing

Which capability is a defining feature of generative AI compared with rule-based test data generators?

  1. Enforcing referential integrity constraints in a test database
  2. Executing test scripts faster than a compiled test runner
  3. Producing novel content such as natural-language scenarios from learned patterns Correct answer.
  4. Guaranteeing byte-identical output for identical inputs

Explanation

Generative models learn statistical patterns from training data and produce new content, including natural-language scenarios and varied inputs. Rule-based generators follow explicit logic and do not create novel content in this sense.

Question 33Fundamentals of Generative AI for Testing

In generative AI terminology used for testing, what is a 'foundation model'?

  1. A model trained exclusively on one organisation's defect records
  2. A large pre-trained model that can be adapted to many downstream tasks Correct answer.
  3. A reference implementation used as the test oracle
  4. A small model embedded in a test framework to assert results

Explanation

A foundation model is trained on broad data at scale and can be adapted through prompting, retrieval or fine-tuning to many tasks, including test design support. It is not tied to a single organisation's data unless adapted.

Question 34Fundamentals of Generative AI for Testing

A tester wants generated test cases to follow the organisation's mandatory template of ID, preconditions, steps and expected results. Which prompting approach most directly supports this?

  1. Increasing the temperature so the model explores more formats
  2. Shortening the prompt to reduce token usage
  3. Asking the model to choose whichever format it finds clearest
  4. Providing the template structure and a filled example in the prompt as a format specification Correct answer.

Explanation

Supplying the required structure plus a worked example constrains the output format and gives the model a concrete pattern to imitate. Higher temperature or shorter prompts increase variance and reduce adherence to the template.

Question 35Fundamentals of Generative AI for Testing

Why does the choice of tokenizer matter when a team estimates whether a long requirements document will fit into a model's context window?

  1. Tokenizers compress text losslessly so the document always fits
  2. Different tokenizers split the same text into different numbers of tokens, changing the effective size Correct answer.
  3. Tokenizers convert text into images, which are not counted against the limit
  4. Tokenizers guarantee that each word maps to exactly one token

Explanation

Token counts depend on the tokenizer's vocabulary and segmentation rules, so the same document can consume different amounts of the context window across models. Estimating by word count alone can therefore misjudge whether the content fits.

Question 36Fundamentals of Generative AI for Testing

A team uses retrieval augmentation so that generated test cases cite internal requirement clauses. What does the retrieval component contribute?

  1. It replaces the need for human review of the generated cases
  2. It retrains the model weights on the internal documents
  3. It supplies relevant document passages as grounding context in the prompt Correct answer.
  4. It stores the model's outputs for later regression comparison

Explanation

Retrieval fetches relevant passages from an external store and inserts them into the prompt, grounding generation in current source material without changing model weights. Human review is still required because grounding reduces but does not eliminate fabrication.

Question 37Fundamentals of Generative AI for Testing

Which risk is most directly associated with sending confidential requirement specifications to a public generative AI service?

  1. Disclosure of sensitive information to a third party outside organisational control Correct answer.
  2. Immediate failure of the test environment database
  3. Loss of the tokenizer's vocabulary mappings
  4. Permanent corruption of the model weights by the submitted text

Explanation

Submitting confidential content to an external service may expose it to a provider whose retention and usage policies are outside the organisation's control. Data-handling agreements and approved deployment options are the usual mitigations.

Question 38Fundamentals of Generative AI for Testing

A team wants a model to answer questions about their internal test process documentation, which changes weekly. Which approach best balances accuracy with maintenance effort?

  1. Manually editing the prompt to paste the whole documentation each time
  2. Retrieval augmentation over the current documentation Correct answer.
  3. Fine-tuning the model each week on the full document set
  4. Relying on the model's pre-trained knowledge of testing standards

Explanation

Retrieval keeps answers grounded in the current documents because the index can be refreshed without retraining. Fine-tuning weekly is costly and still risks staleness, while the model's pre-trained knowledge cannot contain organisation-specific process details.

Question 39Fundamentals of Generative AI for Testing

What is the primary purpose of a 'system prompt' when configuring a generative AI assistant for test design?

  1. To encrypt the conversation between client and server
  2. To store the model's learned parameters between requests
  3. To set persistent role, constraints and output conventions for the session Correct answer.
  4. To define the database schema used by the test environment

Explanation

A system prompt establishes standing instructions such as role, tone, constraints and formatting that apply across the session. It does not alter model parameters or environment configuration.

Question 40Fundamentals of Generative AI for Testing

A tester asks a model to generate boundary values for an age field accepting 18 to 65 inclusive. The response lists 17, 18, 65 and 66. How should the tester treat this output?

  1. Assume the model has executed the tests and confirmed the boundaries
  2. Reject it because generated boundary values are never reliable
  3. Adopt it without review because boundary analysis is deterministic
  4. Accept it as a candidate set and verify completeness against the specification, including any missing non-boundary cases Correct answer.

Explanation

The output is a plausible candidate set, but the tester must confirm it against the specification and consider other cases such as invalid types or empty input. Generated candidates support analysis; they do not replace verification or execution.

Question 41Fundamentals of Generative AI for Testing

Which statement correctly distinguishes fine-tuning from prompting for a testing assistant?

  1. Fine-tuning and prompting both leave model parameters unchanged
  2. Fine-tuning supplies instructions at inference time, while prompting changes model parameters
  3. Fine-tuning changes model parameters using example data, while prompting supplies instructions at inference time Correct answer.
  4. Fine-tuning only affects the tokenizer, while prompting only affects the temperature

Explanation

Fine-tuning adjusts the model's weights through additional training on examples, whereas prompting conditions an unchanged model at inference time. This distinction drives cost, maintenance and how quickly behaviour can be changed.

Question 42Fundamentals of Generative AI for Testing

A generated test suite references requirement identifiers that do not exist in the requirements repository. Which quality attribute of the generated artefact has most clearly failed?

  1. Performance
  2. Portability
  3. Usability
  4. Traceability Correct answer.

Explanation

Traceability requires that test artefacts link to valid, verifiable sources such as requirement identifiers. Referencing non-existent identifiers breaks that link and undermines coverage analysis and change impact assessment.

Question 43Fundamentals of Generative AI for Testing

Which practice best mitigates the risk that a generative assistant produces test steps that look correct but do not match the actual user interface?

  1. Increasing the number of generated alternatives before choosing one
  2. Lowering the temperature to zero and skipping review
  3. Asking the model to state that its output is accurate
  4. Executing the generated steps against the real system and reviewing failures Correct answer.

Explanation

Only execution against the real system reveals mismatches between plausible text and actual behaviour. Reviewing execution results turns generated steps into verified artefacts rather than accepted assumptions.

Question 44Fundamentals of Generative AI for Testing

Why is recording the exact prompt, model version and parameters alongside generated test artefacts valuable?

  1. It enables reproduction and audit of how the artefact was produced Correct answer.
  2. It guarantees the artefact will remain correct after system changes
  3. It prevents the model from hallucinating on future runs
  4. It removes the need to store the generated artefact itself

Explanation

Provenance metadata lets reviewers reproduce or audit generation and investigate later defects in the artefact. It does not guarantee correctness or prevent hallucination, and it does not replace storing the artefact.

Question 45Fundamentals of Generative AI for Testing

A model is asked to summarise a 200-page specification into test conditions. The output omits several safety-related clauses. Which explanation is most consistent with known model behaviour?

  1. The omitted clauses were removed because they contained too many tokens
  2. The model refused to summarise content it had not been fine-tuned on
  3. The model executed the specification and found the clauses untestable
  4. Summarisation compresses content and can drop low-salience details, so coverage must be checked Correct answer.

Explanation

Summarisation inherently compresses and may treat important but low-salience details as expendable, so systematic coverage checks against the source are needed. The model does not execute specifications or silently delete clauses by token count.

Question 46Fundamentals of Generative AI for Testing

Two teams report that the same generative assistant gives strong results for API test ideas but weak results for embedded firmware test ideas. Which interpretation is most defensible?

  1. API testing is inherently easier than firmware testing for all tools
  2. The assistant has a defect that only affects firmware projects
  3. The firmware team is using the wrong temperature setting
  4. Training data coverage differs by domain, so output quality varies with how well the domain is represented Correct answer.

Explanation

Model quality depends heavily on how well the task domain is represented in training and grounding data, which explains uneven performance across domains. Concluding a tool defect or blaming a single parameter ignores this data-coverage explanation.

Question 47Fundamentals of Generative AI for Testing

An organisation wants every AI-generated test artefact to be attributable to a named reviewer before use. Which combination of practices best satisfies this requirement?

  1. Automatic merging of generated artefacts without review to save time
  2. Higher temperature plus longer prompts for each generation
  3. Restricting generation to a single model version indefinitely
  4. Provenance metadata for generation plus a recorded human approval step in the workflow Correct answer.

Explanation

Attribution requires knowing how and by whom an artefact was produced and approved, which provenance metadata plus a recorded approval step provides. Neither prompt tuning nor freezing a model version establishes human accountability.

Question 48Fundamentals of Generative AI for Testing

A team notices that generated test cases for a tax calculation are accepted by reviewers but later prove wrong when regulations change. Which control most directly addresses this class of failure?

  1. Asking the model to regenerate cases with a higher temperature
  2. Storing only the final generated cases without their prompts
  3. Increasing the number of generated cases per requirement
  4. Linking generated cases to versioned source requirements and re-validating on change Correct answer.

Explanation

The failure is staleness relative to changing source material, so linking artefacts to versioned requirements and re-validating on change keeps them aligned. Volume, sampling variation and reduced provenance do not address the underlying drift.

Question 49Fundamentals of Generative AI for Testing

A tester must decide whether a generated set of equivalence partitions for a currency field is usable. Which evaluation approach is most rigorous?

  1. Compare the partitions against the specification and check that valid, invalid and boundary classes are covered Correct answer.
  2. Confirm that the model produced the same partitions twice in a row
  3. Count how many partitions were generated and prefer the largest set
  4. Check that the generated text is grammatically well formed

Explanation

Rigour comes from checking the generated partitions against the specification and confirming coverage of valid, invalid and boundary classes. Fluency, repetition and volume are not evidence of analytical correctness.

Question 50Fundamentals of Generative AI for Testing

Which statement best explains why a generative model can produce a convincing but incorrect expected result for a complex pricing rule?

  1. The model deliberately introduces errors to test the reviewer's attention
  2. The model predicts likely text rather than computing the rule from the authoritative specification Correct answer.
  3. The model's tokenizer corrupts numeric values during encoding
  4. The model has executed the pricing engine and misread its output

Explanation

Language models generate probable sequences rather than evaluating business rules, so a plausible-looking value may not follow the specification. Expected results for such rules must be derived independently and verified.