Retail Peak Latency Despite App Autoscaling
ShopSphere is a web retailer running a three-tier app on one IaaS region: two load-balanced app servers, a primary MySQL database, and one read replica. At peak, traffic is 500 requests per second and p95 latency is 3.2 seconds. App CPU is 35%; database CPU is 55%; database connections are at 95% of the 500-connection limit. The app opens a new database connection per request and does not reuse connections. The team autoscales app servers from two to six, but latency does not improve and sometimes worsens. The read replica is used only for nightly reports. The CTO proposes a larger database instance. The next sale is in 10 days, and the team can change application configuration but not rewrite the data access layer. There is a Redis cache available but unused.
1. Why does adding app servers fail to improve latency? Use the numbers.
Read the worked answer
Adding app servers cannot help because the app tier is not saturated: CPU is 35%. The database connection pool is at 95% of 500, so requests queue for connections. Because the app opens a new connection per request, each added app server increases connection churn and contention, which explains why latency sometimes worsens. Database CPU at 55% is not the immediate ceiling either; connection exhaustion and queueing happen before CPU saturation. The unused read replica means read traffic is still concentrated on the primary. The bottleneck is the connection path, not app compute.
2. Recommend two changes that fit the 10-day window and do not rewrite the data access layer. Explain the mechanism.
Read the worked answer
Change 1: introduce a connection pooler between the app and MySQL-either built-in pooling configuration or a proxy-so the app reuses a bounded number of connections. This caps connections below 500 and removes per-request connection overhead; it is configuration, not a data-layer rewrite. Change 2: enable Redis caching for hot product/catalog reads, with a short TTL and cache-aside pattern. This reduces primary reads and connection hold time without changing the data access layer. A third option is routing existing read-only report queries to the replica, but that is separate. Both fit 10 days if applied to the highest-traffic paths and monitored.
3. What risk does the larger database instance introduce, and what would you monitor after the changes?
Read the worked answer
A larger database instance may raise the connection limit and CPU, but it does not fix connection churn; the app can still exhaust the new limit, and vertical scaling has a ceiling and higher cost. It may also mask the need for pooling until the next sale. After changes, monitor: active connections and wait time, p95 latency, error rate, cache hit ratio, replica lag if reads move, and database CPU. Set alerts at 80% of the pooled connection limit. Confirm the sale traffic with a load test before the event; if p95 does not fall, the next step is query analysis, not another instance size.