ISTQB CT-UT cheat sheet

Formative testing runs during design to find problems and shape the product; summative testing runs against defined metrics to judge whether usability goals were met.

Moderated usability testing has a facilitator who probes, prompts, and asks follow-up questions; unmoderated testing is automated and remote, giving larger samples but no probing.

Think-aloud protocol asks participants to verbalize thoughts while working; retrospective probing (video cued recall) avoids the interference think-aloud causes on complex tasks.

Heuristic evaluation is an expert inspection against recognized principles (e.g., Nielsen's ten heuristics) performed by 3-5 independent evaluators, then consolidated and severity-rated.

Cognitive walkthrough evaluates learnability by stepping through each task action asking: will the user try the right action, notice it, interpret feedback correctly, and progress?

Usability testing sample sizes are chosen for problem discovery, not statistics: small samples (about 5 per user group) find most high-frequency problems; metrics-based studies need larger samples.

A/B testing compares two live variants with real users and a controlled metric; it measures preference/performance but does not explain causes, so pair it with qualitative methods.

Usability test sessions follow a script: welcome and consent, warm-up questions, task scenarios, post-task ratings (e.g., Single Ease Question), debrief; the facilitator avoids leading cues.

Pilot the session with a colleague on the real environment before running participants; the pilot validates tasks, timing, recording, and the consent script.

Separate observation room from the test room; use one-way mirror or video relay so observers do not influence participants; mute observer audio into the test room.

Recording set-up: screen capture plus participant camera plus microphone; verify audio levels and that no notification pop-ups from the facilitator machine appear on the captured screen.

Reset the device to a clean state between participants: clear cookies, history, cached credentials, test accounts, and any prior session data that could bias the next run.

Provide realistic test data and seeded accounts; tasks must not fail because of empty databases, missing permissions, or network restrictions.

Prepare the environment checklist: device, browser/OS version matching the target profile, assistive technology installed and configured, network throttling if relevant, backup device.

Consent and privacy set-up: informed consent form, recording permission, data retention and anonymization rules, and a way to stop recording on request.

Moderator kit: script, task cards, stopwatch or timing tool, severity/observation forms, incentives, and a printed backup of the script if the tooling fails.

Remote sessions: verify the participant's connection, screen sharing, and recording rights at the start; have a fallback phone channel if the platform drops.

Time-boxing: allocate buffer between sessions for reset, note consolidation, and observer debrief; late sessions degrade data quality.

UX spans usefulness, usability, desirability, accessibility, and trust; usability is a subset focused on effectiveness, efficiency, and satisfaction in context of use.

Effectiveness = task completion rate; efficiency = time on task or effort per completed task; satisfaction = post-test questionnaire scores such as SUS or UMUX-Lite.

System Usability Scale (SUS) yields a 0-100 score from ten items; scores above about 68 are commonly treated as above average, and it is a relative, not absolute, measure.

Mental models and expectations drive perceived ease of use: match the interface's language and flow to the user's domain vocabulary and existing habits.

Cognitive load reduction: chunk information, use recognition over recall, provide progressive disclosure, and keep feedback immediate for every user action.

Error prevention and recovery: constrain inputs, confirm destructive actions, provide undo, and write error messages that state the cause and the next step.

Context of use matters: device, environment, interruptions, and user expertise change what counts as good UX; test with representative users in representative settings.

Perceived performance affects UX: show progress, skeleton states, and optimistic updates; response-time thresholds (roughly 0.1s, 1s, 10s) shape perceived responsiveness.

Emotional and trust factors (visual credibility, consistency, transparency about data use) are UX quality attributes that usability metrics alone may miss.

WCAG principles: Perceivable, Operable, Understandable, Robust (POUR); each has testable success criteria at levels A, AA, and AAA.

Common AA targets: text contrast at least 4.5:1 (3:1 for large text), non-text contrast 3:1, visible focus indicator, keyboard operability, and reflow at 320 CSS px width.

Screen reader testing requires pairing a specific browser with a specific assistive technology (e.g., NVDA with Firefox/Chrome, VoiceOver with Safari); results are combination-specific.

Keyboard-only testing: reach and operate every control with Tab, Shift+Tab, Enter, Space, arrow keys; check logical focus order, no keyboard traps, and skip links.

Accessible names and roles: every interactive element needs a programmatic name; images need meaningful alt text or empty alt when decorative; form fields need labels.

Zoom and reflow: test at 200% zoom and 400% reflow; content must not require two-dimensional scrolling except for tables, maps, and similar exceptions.

Motion and timing: respect prefers-reduced-motion, allow pausing of moving content, and warn or allow extension of time limits.

Accessibility is usability for people with disabilities: involve users with disabilities in testing rather than relying only on automated checkers, which catch only a subset of issues.

Automated accessibility tools detect a minority of issues; manual checks plus assistive-technology testing are required to claim conformance.

ISO 9241-11 defines usability as the extent to which a system is used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.

ISO 9241-210 covers human-centred design for interactive systems: understand context, specify user requirements, produce design solutions, evaluate against requirements, iterate.

ISO/IEC 25010 (SQuaRE) defines product quality characteristics including usability and accessibility as quality attributes with sub-characteristics.

WCAG 2.2 (W3C) is the reference standard for web accessibility; EN 301 549 is the European harmonized standard that references WCAG for ICT procurement.

Section 508 (US federal) and the European Accessibility Act are regulations that reference technical standards; distinguish the law from the technical standard it points to.

ISO 9241-110 lists interaction principles (suitability for the task, self-descriptiveness, conformity with expectations, learnability, controllability, error tolerance, individualization).

ISO/IEC 25062 (CIF) and ISO 9241-11 provide the framework for reporting usability test results against metrics; ISO 20282 covers ease of operation of everyday products.

WAI-ARIA specifies roles, states, and properties for custom widgets; use native HTML semantics first and ARIA to fill gaps, not to replace them.

ISTQB usability testing treats standards as the basis for review checklists and for defining measurable usability and accessibility requirements.

Usability risk = likelihood of a usability problem occurring multiplied by its impact on users and the business (lost tasks, errors, support cost, abandonment).

Risk identification sources: heuristic evaluation, user research, analytics (funnels, drop-off, rage clicks), support tickets, and accessibility audits.

Common high-impact risks: unclear navigation, missing feedback, destructive actions without confirmation, inaccessible controls, and inconsistent terminology.

Severity rating combines frequency, impact, and persistence of a problem; a rare but catastrophic failure (data loss) outranks a frequent cosmetic issue.

Prioritize risks by user criticality: failures on safety-critical or financial tasks carry higher business and legal risk than cosmetic defects.

Accessibility failures are legal and reputational risks in regulated markets; procurement often requires demonstrated WCAG conformance.

Track usability risks in the same defect/risk system as functional risks so they are visible to project management and prioritized alongside them.

Mitigation options: redesign, add guidance or defaults, improve error recovery, provide alternative paths, or accept with documented rationale.

Re-test after mitigation to confirm the risk is reduced; a fixed defect that is not re-tested remains an open risk.

Formative testing runs during design to find problems and shape the product; summative testing runs against defined metrics to judge whether usability goals were met.

Moderated usability testing has a facilitator who probes, prompts, and asks follow-up questions; unmoderated testing is automated and remote, giving larger samples but no probing.

Think-aloud protocol asks participants to verbalize thoughts while working; retrospective probing (video cued recall) avoids the interference think-aloud causes on complex tasks.

Heuristic evaluation is an expert inspection against recognized principles (e.g., Nielsen's ten heuristics) performed by 3-5 independent evaluators, then consolidated and severity-rated.

Cognitive walkthrough evaluates learnability by stepping through each task action asking: will the user try the right action, notice it, interpret feedback correctly, and progress?

Usability testing sample sizes are chosen for problem discovery, not statistics: small samples (about 5 per user group) find most high-frequency problems; metrics-based studies need larger samples.

A/B testing compares two live variants with real users and a controlled metric; it measures preference/performance but does not explain causes, so pair it with qualitative methods.

Usability test sessions follow a script: welcome and consent, warm-up questions, task scenarios, post-task ratings (e.g., Single Ease Question), debrief; the facilitator avoids leading cues.

Pilot the session with a colleague on the real environment before running participants; the pilot validates tasks, timing, recording, and the consent script.

Separate observation room from the test room; use one-way mirror or video relay so observers do not influence participants; mute observer audio into the test room.

Recording set-up: screen capture plus participant camera plus microphone; verify audio levels and that no notification pop-ups from the facilitator machine appear on the captured screen.

Reset the device to a clean state between participants: clear cookies, history, cached credentials, test accounts, and any prior session data that could bias the next run.

Provide realistic test data and seeded accounts; tasks must not fail because of empty databases, missing permissions, or network restrictions.

Prepare the environment checklist: device, browser/OS version matching the target profile, assistive technology installed and configured, network throttling if relevant, backup device.

Consent and privacy set-up: informed consent form, recording permission, data retention and anonymization rules, and a way to stop recording on request.

Moderator kit: script, task cards, stopwatch or timing tool, severity/observation forms, incentives, and a printed backup of the script if the tooling fails.

Remote sessions: verify the participant's connection, screen sharing, and recording rights at the start; have a fallback phone channel if the platform drops.

Time-boxing: allocate buffer between sessions for reset, note consolidation, and observer debrief; late sessions degrade data quality.

UX spans usefulness, usability, desirability, accessibility, and trust; usability is a subset focused on effectiveness, efficiency, and satisfaction in context of use.

Effectiveness = task completion rate; efficiency = time on task or effort per completed task; satisfaction = post-test questionnaire scores such as SUS or UMUX-Lite.

System Usability Scale (SUS) yields a 0-100 score from ten items; scores above about 68 are commonly treated as above average, and it is a relative, not absolute, measure.

Mental models and expectations drive perceived ease of use: match the interface's language and flow to the user's domain vocabulary and existing habits.

Cognitive load reduction: chunk information, use recognition over recall, provide progressive disclosure, and keep feedback immediate for every user action.

Error prevention and recovery: constrain inputs, confirm destructive actions, provide undo, and write error messages that state the cause and the next step.

Context of use matters: device, environment, interruptions, and user expertise change what counts as good UX; test with representative users in representative settings.

Perceived performance affects UX: show progress, skeleton states, and optimistic updates; response-time thresholds (roughly 0.1s, 1s, 10s) shape perceived responsiveness.

Emotional and trust factors (visual credibility, consistency, transparency about data use) are UX quality attributes that usability metrics alone may miss.

WCAG principles: Perceivable, Operable, Understandable, Robust (POUR); each has testable success criteria at levels A, AA, and AAA.

Common AA targets: text contrast at least 4.5:1 (3:1 for large text), non-text contrast 3:1, visible focus indicator, keyboard operability, and reflow at 320 CSS px width.

Screen reader testing requires pairing a specific browser with a specific assistive technology (e.g., NVDA with Firefox/Chrome, VoiceOver with Safari); results are combination-specific.

Keyboard-only testing: reach and operate every control with Tab, Shift+Tab, Enter, Space, arrow keys; check logical focus order, no keyboard traps, and skip links.

Accessible names and roles: every interactive element needs a programmatic name; images need meaningful alt text or empty alt when decorative; form fields need labels.

Zoom and reflow: test at 200% zoom and 400% reflow; content must not require two-dimensional scrolling except for tables, maps, and similar exceptions.

Motion and timing: respect prefers-reduced-motion, allow pausing of moving content, and warn or allow extension of time limits.

Accessibility is usability for people with disabilities: involve users with disabilities in testing rather than relying only on automated checkers, which catch only a subset of issues.

Automated accessibility tools detect a minority of issues; manual checks plus assistive-technology testing are required to claim conformance.

ISO 9241-11 defines usability as the extent to which a system is used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.

ISO 9241-210 covers human-centred design for interactive systems: understand context, specify user requirements, produce design solutions, evaluate against requirements, iterate.

ISO/IEC 25010 (SQuaRE) defines product quality characteristics including usability and accessibility as quality attributes with sub-characteristics.

WCAG 2.2 (W3C) is the reference standard for web accessibility; EN 301 549 is the European harmonized standard that references WCAG for ICT procurement.

Section 508 (US federal) and the European Accessibility Act are regulations that reference technical standards; distinguish the law from the technical standard it points to.

ISO 9241-110 lists interaction principles (suitability for the task, self-descriptiveness, conformity with expectations, learnability, controllability, error tolerance, individualization).

ISO/IEC 25062 (CIF) and ISO 9241-11 provide the framework for reporting usability test results against metrics; ISO 20282 covers ease of operation of everyday products.

WAI-ARIA specifies roles, states, and properties for custom widgets; use native HTML semantics first and ARIA to fill gaps, not to replace them.

ISTQB usability testing treats standards as the basis for review checklists and for defining measurable usability and accessibility requirements.

Usability risk = likelihood of a usability problem occurring multiplied by its impact on users and the business (lost tasks, errors, support cost, abandonment).

Risk identification sources: heuristic evaluation, user research, analytics (funnels, drop-off, rage clicks), support tickets, and accessibility audits.

Common high-impact risks: unclear navigation, missing feedback, destructive actions without confirmation, inaccessible controls, and inconsistent terminology.

Severity rating combines frequency, impact, and persistence of a problem; a rare but catastrophic failure (data loss) outranks a frequent cosmetic issue.

Prioritize risks by user criticality: failures on safety-critical or financial tasks carry higher business and legal risk than cosmetic defects.

Accessibility failures are legal and reputational risks in regulated markets; procurement often requires demonstrated WCAG conformance.

Track usability risks in the same defect/risk system as functional risks so they are visible to project management and prioritized alongside them.

Mitigation options: redesign, add guidance or defaults, improve error recovery, provide alternative paths, or accept with documented rationale.

Re-test after mitigation to confirm the risk is reduced; a fixed defect that is not re-tested remains an open risk.