Evaluating Security Tests from Risk, Requirements, Vulnerability and Human-Factors Perspectives
Evaluating Security Tests
Evaluation questions dominate the exam: given results, is the system acceptable, and what evidence supports that judgement? You need four lenses, not a single count of findings.
Core rules
- Risk lens: score each finding as likelihood × impact, adjusted for asset value and existing controls; compare residual risk with the organisation's risk appetite and with the threats in the threat model.
- Requirements lens: traceability. Every testable security requirement and standard clause (for example an OWASP ASVS level) should map to at least one test, and every test back to a requirement; uncovered requirements represent unknown risk, not zero risk.
- Vulnerability lens: classify weaknesses (CWE), score them (CVSS base, then temporal and environmental adjustment), remove duplicates and false positives, and consider the vulnerability management lifecycle and remediation SLAs.
- Human-factors lens: people are part of the attack surface - phishing susceptibility, password reuse, insider misuse, and usability of controls. If multi-factor authentication is cumbersome, users will bypass it; measure awareness with consented simulations and usability review.
- Useful metrics: findings by severity, requirement coverage, mean time to remediate, reopened defects - never a single "number of vulnerabilities".
Worked example
A release shows zero critical, three high and twelve medium findings; requirement coverage is 85%; a phishing simulation produced a 30% click rate. Two high findings concern authentication and have no compensating control, and the uncovered 15% includes logging and monitoring. The judgement is that residual risk exceeds appetite: release is blocked, the authentication findings are fixed and retested, monitoring requirements are tested, and awareness training is repeated before the simulation is rerun. The reasoning is risk-based, not count-based.
Traps
Reading "no critical findings" as "secure"; ignoring false negatives; rating risk with no business context; dismissing human factors as untestable; using a raw CVSS score as the final risk decision.