ISTQB CT-PT cheat sheet

Performance testing is a dynamic, non-functional test type that measures speed, capacity, and stability behavior of a system under specified load conditions; it verifies characteristics, not functional correctness.

The four classic performance test types: load test (expected/peak load), stress test (beyond limits to find breaking point), soak/endurance test (sustained load over long duration), spike test (sudden sharp load increase).

Scalability testing determines whether the system can handle growth in users, data volume, or transactions; it is about the ability to scale up or out, distinct from raw capacity measurement.

Performance efficiency (ISO/IEC 25010) has three sub-characteristics: time behavior, resource utilization, and capacity - a defect against any of these is a performance defect.

A performance defect is a deviation where response time, throughput, or resource use fails to meet a stated requirement; root causes often lie outside the tested application (network, DB, third-party services, infrastructure).

Load is not equal to concurrency: concurrent users may generate different request rates depending on think time; load must be modeled in terms of transactions per unit time or arrival rate, not just user count.

Baseline testing captures a known reference measurement of a stable build; it is used later to detect regressions by comparison, so the baseline environment and workload must be identical.

Risk-based selection: performance testing effort is prioritized on business-critical transactions with high usage frequency, high data volume, or strict response-time SLAs.

Response time = time from request sent to last byte of response received; it includes client processing, network latency, server queuing, and server processing - never treat it as server-side processing time alone.

Percentile metrics (e.g., 90th, 95th, 99th) describe the distribution tail; an average response time hides outliers, so SLAs are usually expressed as a percentile (e.g., 95% of requests under 2 s).

Throughput is the number of completed transactions or requests per unit time; Little's Law relates the three core metrics: N (concurrent users) = X (throughput) × R (average response time).

Utilization is the fraction of a resource's capacity in use; queuing theory shows response time rises sharply as utilization approaches 100%, so headroom (e.g., CPU below ~70-80%) is a common operating rule.

Think time is the pause between a user's consecutive requests; reducing think time increases generated load, so think time must be modeled and recorded with the workload profile.

Measurement errors: client-side timers include network and client overhead; server-side timers exclude them; clock synchronization and load-generator bottlenecks distort both - validate the load generator is not the bottleneck.

Resource metrics to collect: CPU, memory, disk I/O, network bandwidth, thread/connection pool usage, garbage collection, and database wait events; correlate them with response-time changes over time.

Hit rate and cache statistics affect measured performance; a test run with a cold cache measures different behavior than a warm-cache run, so cache state must be defined and reported.

Activity sequence: define performance requirements and acceptance criteria → plan → design workload model → implement scripts → configure and validate environment → execute → monitor and analyze → report → retest/regression.

Workload modeling defines user types, transaction mix (percentages), arrival pattern (open vs closed model), think times, and test data; it must reflect production usage, not an arbitrary user count.

Script development tasks: record or code the protocol-level requests, parameterize input data, add transactions/timers, add think time, implement validation checks, and correlate dynamic values (session IDs, tokens).

Correlation is the task of capturing server-generated dynamic values from a response and reusing them in subsequent requests; missing correlation causes failed or unrealistic script execution.

Test environment readiness: production-like hardware, network, data volume, and configuration; the environment is a validity threat if it differs from production, and results must be qualified accordingly.

Execution tasks: smoke test the script, run a baseline, then ramp to target load; monitor in parallel and record timestamps so metric spikes can be correlated with load changes.

Analysis tasks: compare against acceptance criteria, identify the bottleneck (client, network, app, DB, infrastructure), and separate symptoms (high response time) from causes (e.g., lock contention).

Reporting must state the workload, environment, metrics, and pass/fail per criterion; results without the workload model and environment description are not reproducible.

Regression performance testing re-runs an established workload after changes to detect degradation relative to the baseline; it requires stable scripts and comparable environments.

Tool categories: load-generation tools (protocol-level, generate virtual users), monitoring tools (APM, OS, network, DB), profiling tools (code-level), and analysis/reporting tools; no single tool covers all four.

Protocol-level (headless) tools simulate many virtual users with low resource cost; browser-based tools measure real rendering and client-side time but scale to far fewer users.

Virtual users are simulated users executed by threads or processes; the tool's own resource consumption sets a ceiling, so multiple load generators are often required for high load.

Tool selection criteria: supported protocols, scripting and correlation capability, monitoring integration, license/cost, scalability, reporting, and skill availability in the team.

Licensing models differ (per virtual user, per protocol, perpetual vs subscription); cost scales with the maximum concurrent user count, so license planning is part of test planning.

Organizational roles: performance test lead/architect (strategy, workload model), performance tester/engineer (scripts, execution), performance analyst (results interpretation), and stakeholders (requirements, sign-off).

Early performance testing (shift-left) uses component-level or API-level tests in CI to catch regressions cheaply; full-scale load tests remain periodic because they need production-like environments.

Non-production environments are usually smaller than production; results must be extrapolated cautiously and the extrapolation assumptions documented in the report.

Performance testing requires cross-team coordination: development (fixes, profiling), operations/infrastructure (environment, monitoring), DBAs (query tuning), and business (workload validity, acceptance criteria).