Non-Functional Testing: Performance, Compatibility, and Usability
What this domain tests
Non-functional testing covers qualities rather than features: frame rate and frame time, memory, load times, thermal behaviour, hardware and software compatibility, and usability. The exam expects you to choose measurable targets and to test the configurations players actually own.
Core rules
- Frame budget: 60 fps allows about 16.7 ms per frame; 30 fps allows 33.3 ms. Report frame-time percentiles (1% and 0.1% lows), not only average FPS, because stutter hides in the tail.
- Measure on representative hardware: minimum spec, recommended spec, a current console, a launch-model console, handheld mode, and at least one unusual aspect ratio.
- Compatibility matrix: OS versions, GPU drivers, CPU families, storage type, resolution, input device, language and region.
- Load-time and memory budgets should be stated per level or scene, with a tolerance and a repeatable measurement method.
- Usability: onboarding, HUD readability, subtitle size, colour-blind modes, remapping, difficulty options and text scaling.
Worked example
On the minimum-spec PC, average frame rate is 58 fps, which passes the 60 fps target within tolerance. The frame-time graph shows 1% lows of 60 ms during combat, so the build hitches. Reasoning: the average hides a visible stutter; the correct comparison is the 99th-percentile frame time against the budget, not the mean. The tester repeats the run three times on the same save to rule out background processes, captures a profile, and files the defect with the scene, hardware, driver version and the percentile figures. The pass verdict is withdrawn.
Traps
Judging performance from an on-screen FPS counter alone. Single runs with no repeatability. Testing only on a development kit or a high-end PC. Ignoring thermal throttling on handhelds and laptops. Treating compatibility as 'it launches'. Leaving accessibility checks until the final week.