Validating a 2,000-User Peak Requirement with Little's Law
A travel-booking platform must meet a peak requirement: 2,000 concurrent online users, 95th-percentile page response <= 2.0 s, error rate < 0.5%, sustained for 30 minutes. The performance team scripted the booking journey in JMeter with 2,000 threads, no timers, and ran a single 30-minute test. Results: throughput 1,820 requests/s, average response 1.08 s, 95th percentile 4.6 s, 2.3% errors, all HTTP 500s logged as 'could not get connection from pool within 3000 ms'. Application server CPU averaged 95%; database server CPU 45%. The connection pool is fixed at 40 connections per instance, with two instances behind the load balancer; average database service time per request is 60 ms. The test rig reported 60% CPU. The project manager reads the summary and declares the release blocked by 'a database capacity problem'; the infrastructure team wants to raise the pool to 200 per instance.
1. Determine whether this test is a valid measurement of the stated requirement, showing the arithmetic.
Read the worked answer
No. The numbers show the script had no think time, so the closed model is running at its maximum, not at a realistic user pace.
Little's law for a closed workload: throughput = users / (response time + think time).
- Observed: 2,000 / 1.08 s = 1,852, which matches the reported 1,820 within rounding. Think time is effectively zero.
- A booking user typically reads a page for about 6 s. With that pace: 2,000 / (1.08 + 6) = 2000 / 7.08 = 283 requests/s.
The test therefore drove roughly 6.4 times the request rate a real population of 2,000 users would generate. Every response-time and error figure describes an overload condition, not the requirement. Two further validity points: the 95th percentile of 4.6 s against an average of 1.08 s means the delay is queuing-based, not computation-based; and a single 30-minute run with no baseline or soak cannot show whether performance degrades over time.
The correct conclusion is 'requirement not yet measured', not 'release blocked'. The team must first define what 2,000 concurrent users means operationally (sessions with think time) and separately derive a target arrival rate from production logs, typically the busiest hour plus a growth margin.
2. Is the connection pool the root cause? Compute the connection demand and assess the proposal to raise the pool to 200.
Read the worked answer
Required concurrent connections follow Little's law again: connections = throughput x database service time.
- At the measured 1,820 req/s spread over two instances, each instance serves about 910 req/s. Demand = 910 x 0.06 s = 54.6 connections, above the 40 available. Requests queue, some wait past the 3,000 ms timeout, and return HTTP 500. The error text is real but it is a symptom of overload.
- At the realistic 283 req/s, each instance serves about 141 req/s, needing 141 x 0.06 = 8.5 connections. The current pool of 40 is more than four times oversized for design load.
So the pool is not a capacity defect; it is a saturation boundary that the invalid test crossed. Raising the pool to 200 would make the symptom disappear at 1,820 req/s while pushing load onto a database already at 45% CPU - masking the problem and storing up a worse failure mode.
Legitimate changes, independent of the rerun:
- Set the pool from arithmetic (peak TPS x service time x safety factor of about 1.5), and document the derivation.
- Reduce the acquisition timeout so it is shorter than the user's page deadline, and fail fast with a handled error or load-shed response rather than a 3,000 ms stall.
- Add a circuit breaker so an exhausted pool does not cascade into unrelated journeys.
3. Design the corrected run plan and say what workload model you would derive it from.
Read the worked answer
Derive the workload model from production data before touching the tool. Extract from access logs, for the 30 busiest days: unique sessions per minute, requests per session, inter-request gaps (the empirical think time), the transaction mix and its arrival distribution. If the platform has no traffic history, state that assumption explicitly and treat the model as a hypothesis to be revised after launch.
Run plan, each with entry criteria and a recorded baseline:
- Baseline: 1 user, 10 minutes - establishes uncontended response time and proves the script and oracle are correct.
- Load test: paced at the derived peak (about 283 req/s, 2,000 sessions with observed think time), 60 minutes, ramped over 10 minutes. Success criteria: p95 <= 2.0 s, error rate < 0.5%, CPU < 75% on both tiers.
- Stress test: step-hold increments of 10% until error rate > 1% or p95 doubles, to locate the knee and the breaking point for capacity planning.
- Spike test: 5x arrival rate for 2 minutes from steady state, to verify recovery and that no thread pool or queue stays wedged.
- Soak test: 8-24 hours at 70% of peak to expose leaks - the metric is the trend of heap, GC time and response time, not a single average.
Record the workload model, pacing, and rig CPU in the report; a rig above 70% invalidates the measurement. Percentiles, not averages, are the acceptance evidence.