Performance Measurement and Metrics
What this domain tests
This domain covers measurement: response time, throughput, utilization, concurrency, think time, error rates, and percentiles. The exam tests whether you can define a metric precisely and interpret it without being misled by averages or tool defaults.
Core rules and concepts
- Response time is elapsed time from request to complete response; latency is a component delay, often network or queueing.
- Throughput is completed work per unit time (transactions/s, requests/s, bytes/s); it must be tied to a transaction definition.
- Utilization is the percentage of resource capacity in use (CPU, memory, disk, network); high utilization can cause queueing and nonlinear response-time growth.
- Concurrency is the number of simultaneous users or requests; think time is the pause between requests; pacing controls the interval between iterations.
- Percentiles describe the distribution: p95 means 95% of samples are at or below that value. Averages hide tails.
- Little's Law for a stable system: L = λ × W (items in system = arrival rate × time in system). In a closed user model, throughput ≈ concurrency / (response time + think time).
Worked example
A test records 1,000 requests: 940 complete in 200 ms and 60 complete in 3,000 ms. Average = (940×200 + 60×3,000) / 1,000 = 368 ms. The nearest-rank p95 is the 950th value, which falls in the slow group, so p95 = 3,000 ms. The average suggests acceptable speed, but 6% of users wait 3 s. The correct conclusion: the tail violates a 2 s p95 target; investigate the slow cohort, not the mean.
Traps
Averaging percentiles across runs; comparing p95 values from different sample sizes without noting it; ignoring think time when calculating concurrency; measuring only server time while users experience network plus client time; treating hits as transactions; reporting throughput without error rate.