ISTQB CT-UT common mistakes
ISTQB CT-UT · Understand the traps and how to avoid them.
Treating usability testing as the same activity as functional testing: writing test cases that only check whether a feature works, then labelling the result a usability finding.
Candidates prepare with functional-testing habits, where pass/fail against a specification is the goal. Usability testing measures effectiveness, efficiency, and satisfaction for real users, and a feature can work perfectly while being unusable.
What to do instead
For each candidate test, state the usability attribute being evaluated (effectiveness, efficiency, satisfaction, freedom from risk, or context coverage from ISO 25010/9241-11) and the user group and task context. If you cannot name the user, the task, and the measurable outcome, it is not a usability test.
Confusing usability testing with user acceptance testing and answering that UAT validates usability because both involve users.
Both activities involve real users, so the terms blur. UAT confirms the system meets business and contractual requirements; usability testing observes how representative users perform tasks to find interaction problems.
What to do instead
Anchor the distinction on purpose and output: UAT produces acceptance decisions against requirements, usability testing produces observations and defect reports about the user interface and interaction design. State this contrast explicitly when a question asks you to compare techniques.
Choosing a method by preference rather than by question, for example always recommending a large quantitative lab study when the question describes an early prototype with no budget.
Candidates memorise a favourite technique and apply it regardless of the constraints given. Exam scenarios deliberately include stage, budget, participant availability, and fidelity as decision inputs.
What to do instead
Read the scenario for four signals: development stage, prototype fidelity, whether the question is qualitative or quantitative, and resource constraints. Early low-fidelity work points to heuristic evaluation, expert review, or small qualitative sessions; statistically valid metrics need larger samples and higher fidelity.
Describing a usability test session as 'show the user the feature and ask if they like it', omitting task design, moderator neutrality, and think-aloud protocol.
This is the intuitive picture of a demo, not a test. Candidates without session experience default to demonstration and opinion gathering, which produces leading data.
What to do instead
Describe the full session structure: a realistic task scenario with a goal and no step-by-step instructions, a neutral moderator who avoids leading prompts, a think-aloud instruction with a practice task, observation and logging of critical incidents, and a post-session questionnaire or interview. Mention piloting the session.
Skipping or misordering set-up steps: starting data collection before defining participants, environment, tasks, consent, and logging, or treating set-up as purely technical installation.
Set-up is often memorised as a checklist of equipment, so candidates miss that participant recruitment criteria, consent and privacy handling, moderator script, and pilot run are part of usability test set-up.
What to do instead
Structure set-up in layers: define objectives and metrics; define participant profile and recruit; prepare tasks, script, and materials; prepare environment and instrumentation (recording, logging, screen capture); handle consent and data protection; run a pilot and adjust. Recall that a pilot with a colleague catches task wording and equipment faults before real participants arrive.
Ignoring the test environment and data set-up: using unrealistic or empty data, a production-like environment that differs from the target, or forgetting to reset state between participants.
Candidates focus on the interface under test and overlook that data volume, account state, and environment configuration change what participants can do and how long tasks take.
What to do instead
Specify representative data (realistic volume, edge cases, seeded accounts), a defined environment matching the target platform and devices, reset or restore procedures between sessions, and a check that assistive technology and browser or device configurations are correct before each participant.
Reducing user experience to visual appeal, so answers treat UX as colour, layout, and aesthetics only.
Everyday usage of 'UX' has collapsed into look and feel. The exam expects a broader view covering usefulness, usability, desirability, accessibility, and the whole journey including before and after the interaction.
What to do instead
Answer with the multidimensional model: usefulness, usability, desirability, accessibility, credibility, and findability, plus the emotional and temporal dimensions of the experience. When a question asks about UX, mention the user journey and context of use, not just the screen.
Treating accessibility as a compliance checkbox limited to screen readers, and forgetting cognitive, motor, and situational limitations.
Accessibility is often learned as a narrow list of assistive technologies. Candidates then miss questions about keyboard-only operation, contrast, captions, plain language, time limits, and temporary or situational impairments.
What to do instead
Cover the four POUR principles (perceivable, operable, understandable, robust) and map each to concrete checks: text alternatives, keyboard operability and focus order, sufficient contrast and resizable text, consistent navigation and error identification, and compatibility with assistive technology. Include users with cognitive and motor impairments and situational constraints such as bright sunlight or a noisy environment.
Citing the wrong standard or inventing a clause number, for example attributing web content accessibility requirements to ISO 9241 or quoting WCAG success criteria from memory incorrectly.
Several standards overlap in scope and candidates memorise names without their purpose. Guessing clause numbers under time pressure produces confidently wrong answers.
What to do instead
Learn each standard by purpose, not number: ISO 9241 series for ergonomics of human-system interaction (including usability definitions and dialogue principles), WCAG for web content accessibility with its A, AA, AAA levels and version, and ISO 25010 for product quality characteristics including usability and accessibility subcharacteristics. If you cannot recall a clause, answer by principle and do not fabricate a reference.
Listing usability risks generically ('the UI may be confusing') without linking them to user groups, tasks, impact, and likelihood, or omitting usability risks from the risk register entirely.
Risk analysis is usually practised on technical risks, so usability risks are stated as vague opinions and never prioritised, which the exam treats as an incomplete risk analysis.
What to do instead
Express each usability risk as cause, event, and impact for a defined user group and task, for example 'older users cannot complete the checkout because of small touch targets, leading to abandoned orders'. Rate likelihood and impact, assign a mitigation such as a design review or usability test, and record it alongside other product risks so it can be tracked.