Non-Functional Testing: Quality Characteristics Beyond Functionality
Non-Functional Testing: Quality Characteristics Beyond Functionality
This domain tests ISO/IEC 25010 quality characteristics and the design of performance, security, reliability, maintainability, and portability tests. You must turn vague quality expectations into measurable test conditions.
Core rules
- ISO/IEC 25010 characteristics include performance efficiency (time behavior, resource utilization, capacity), reliability (availability, fault tolerance, recoverability), security (confidentiality, integrity, authentication, authorization, non-repudiation), compatibility, usability, maintainability (analyzability, modifiability, testability), and portability (adaptability, installability, coexistence).
- Performance test types: load (expected load), stress (beyond limits), spike (sudden load), endurance/soak (long duration), scalability, and capacity.
- Security testing covers authentication, authorization, input validation, session management, cryptography, and vulnerability scanning; it is not only penetration testing.
- Maintainability is tested through static analysis, reviews, metrics (complexity, duplication), and change impact analysis.
- Portability testing includes install/uninstall, migration, coexistence, and replacement in target environments.
- Non-functional requirements need thresholds: response time percentiles, throughput, availability, recovery time, and resource ceilings.
Worked example
A ticket-sales site must support 10,000 concurrent users, 95th-percentile response under 2 seconds, 99.9% availability, and PCI DSS compliance. The TTA designs a load test at 10,000 users, a stress test to find the breaking point, a 24-hour soak test to expose memory leaks, failover tests for availability, and security tests for injection and broken authentication. Reasoning: each requirement maps to a test type and a measurable exit criterion; a single performance test would not cover capacity, endurance, and recovery.
Traps
- Reporting average response time instead of the 95th percentile.
- Confusing load testing with stress testing.
- Testing performance in a shared or unrepresentative environment.
- Leaving security testing until the end.
- Treating non-functional requirements as untestable because they lack numbers.