QAI CAST common mistakes
QAI CAST · Understand the traps and how to avoid them.
Answering 'testing' questions by describing debugging or code review, e.g. saying that running the program to find the cause of a crash is testing, rather than distinguishing testing (finding failures) from debugging (locating and fixing the defect).
Candidates come from hands-on development backgrounds where 'testing' informally means 'trying it out and fixing it'. They answer from that mental model instead of the exam's vocabulary of failure, fault, and error.
What to do instead
Before the exam, write out one-sentence definitions for error, defect, failure, testing, and debugging, and check each practice question against them. When an option mixes 'find' and 'fix' activities, separate the two steps explicitly before choosing.
Treating verification and validation, or the test levels (unit, integration, system, acceptance), as interchangeable, so an activity such as 'checking the specification is met' is labelled validation.
The words sound similar and everyday use blurs them. Candidates also map test levels to who performs them (developer vs tester) rather than to the object and objective of each level.
What to do instead
Use the mnemonic 'verification = are we building it right (against specification), validation = did we build the right thing (against user need)'. For test levels, memorise the object under test and the typical reference document for each, and classify each scenario by those two attributes only.
Writing test conditions or test cases that contain no expected result, or whose expected result restates the action ('click Save and the record is saved'), making pass/fail undecidable.
Candidates focus on steps because steps feel productive, and they assume the expected outcome is obvious. Exam items that ask whether a case is complete then defeat them.
What to do instead
Adopt a template with a mandatory expected-result column and test it against the question 'could two testers disagree on the outcome?' If yes, the expected result is not specific enough. Rewrite it as an observable, checkable state such as a field value, message text, or stored record count.
Confusing test design techniques with each other, for example calling boundary value analysis 'equivalence partitioning', or claiming decision table testing is a black-box technique that needs code coverage.
The techniques are taught together and share the idea of partitioning inputs, so the labels blur. Candidates also memorise names without practising deriving cases from a small specification.
What to do instead
For each technique, practise on a one-paragraph spec: identify the partitions, then the boundaries (just below, on, just above), then the conditions and actions for a decision table. State the technique's purpose in one line (e.g. boundary value analysis targets the defects most likely at the edges of a partition) so the label follows from the purpose.
Classifying every unexpected behaviour as a defect and raising it, including cases where the test data was wrong, the environment was misconfigured, or the behaviour matches the specification and the tester's expectation was wrong.
Defect-raising feels like the safe, diligent action, and there is pressure to show throughput. Candidates skip the triage step of reproducing and checking against the expected result and specification.
What to do instead
Apply a fixed pre-raise checklist: reproduce once, confirm the expected result from the specification or test basis, rule out data and environment causes, then record actual versus expected with evidence. If the specification is ambiguous, raise a question against the test basis rather than a defect.
Writing defect reports with a title like 'login does not work' and no steps to reproduce, environment, or severity/priority distinction, then choosing 'High' severity and 'High' priority interchangeably.
Under time pressure candidates write the report as a note to themselves rather than for a developer who was not present. Severity and priority are assumed to move together.
What to do instead
Use a fixed report skeleton: summary, environment/build, preconditions, numbered steps, expected, actual, evidence, severity, priority. Learn that severity is impact on the system and priority is urgency of the fix, and practise examples where they diverge (a crash in a rarely used report = high severity, low priority; a misspelled company name on the home page = low severity, high priority).
Interpreting a coverage metric as a quality verdict, e.g. concluding that 100% statement coverage means the code is fully tested, or that a high defect-detection rate in one module proves the test process is effective.
Numbers are reassuring and easy to report. Candidates conflate 'exercised' with 'verified' and forget that coverage measures what was run, not whether the results were checked.
What to do instead
For every metric question, state what the metric measures and what it does not. Pair coverage with defect detection and defect leakage, and remember that 100% statement coverage can still miss unexecuted branches and unchecked assertions. When asked to interpret a metric, name the decision it supports and its limitation.
Confusing defect metrics with each other, for example using defect density (defects per size of component) as if it were defect removal efficiency, or treating a falling open-defect count as proof of improving quality without checking arrival and closure rates.
The formulas look similar and candidates memorise them without units. Trend interpretation is also harder than formula recall, so candidates answer the formula and ignore the question's 'what does this tell you' framing.
What to do instead
Write each metric with its formula and units on one page (defect density = defects / size; defect removal efficiency = defects found before release / total defects found; defect leakage = escaped defects / total defects). For trend questions, ask what else could explain the pattern (fewer tests run, backlog of unverified fixes) before concluding.
Assuming a record-and-playback tool removes the need for test design, or that automating a manual test case unchanged will produce a maintainable automated test.
Tool demos show quick script creation, so candidates equate automation with capture. They also overlook that manual cases often contain redundant steps and human judgement that do not translate directly.
What to do instead
Remember that automation executes designed checks; it does not design them. For each candidate, ask whether it is stable, repeatable, and has a clear pass/fail oracle before automating. Design the automated test around the check, not around the manual script's keystrokes.
Treating test automation as a one-off project rather than a maintained asset, so questions about script maintenance, test data management, and when to retire automated tests are answered as if scripts never need updating.
Candidates have seen automation succeed or fail in one context and generalise. Maintenance cost is invisible in a short exam scenario, so it is easy to ignore.
What to do instead
For any automation question, explicitly consider the full lifecycle: creation, execution in CI, result analysis, maintenance when the application changes, and retirement when the test no longer earns its keep. Prefer options that mention maintainability, reusable test data, and scheduled review over options that promise 'set and forget'.