Promotion spike on a stateless Java tier
A commerce team runs ten Server instances behind an Application Load Balancer in one VPC. Each instance is 4 vCPU and 8 GB, serving a stateless Java API; sessions live in Cloud DB for Redis. Normal CPU is 35%, but a planned 90-minute promotion should triple requests. The image has a 4-minute JVM warm-up and a 2-minute OS boot. Auto Scaling is not enabled. The backend Cloud DB for MySQL accepts at most 300 connections, and the current ten instances already hold 200. Operations wants zero failed requests during scale-out, a predictable cost ceiling, and no code changes. They can change instance type, image, and scaling policy. Cloud Insight shows CPU rises three minutes after traffic. The promotion runs 10:00 to 12:00. They have two weeks to load-test.
1. Design the Auto Scaling policy and database connection plan. Explain why target tracking alone is insufficient and how you keep the 300-connection cap from causing an outage.
Read the worked answer
- Baseline math: ten instances at 35% CPU is 350 CPU-percent units. Tripling load needs roughly 30 instances at the same per-instance CPU. At the current 20 connections per instance, 30 instances demand 600 connections, double the 300 cap.
- Use an Auto Scaling group with min 10 and max 30. Add target tracking on average CPU at 55% with a 180-second cooldown. Add a scheduled action at 09:30 to reach 18 instances and another at 12:15 to return to 10.
- Target tracking alone is insufficient because the metric lags traffic by three minutes and a new JVM needs four more minutes. The promotion is a step change, not a gradual trend, so the first seven minutes would run at overload. Scheduled pre-scaling covers the step; target tracking covers unexpected variance after 10:00.
- Protect the database: raise the Cloud DB for MySQL spec and max_connections to at least 600, or cut the pool to 8 to 10 connections per instance. Prefer both: pool 16 times 30 instances is 480, leaving 120 for admin and batch. Validate the parameter group in staging.
- For zero failed requests, use readiness checks, a health-check grace period longer than JVM warm-up, 60-second connection draining, and 120-second load balancer slow start. Manual scaling is rejected because it depends on a human at 10:00 and cannot react to an unexpected spike.
2. What evidence would you require from the load test before trusting this design, and which Cloud Insight alarms and actions should be wired?
Read the worked answer
- Load test shape: ramp from 1x to 3x in two minutes, hold for 90 minutes, then drop to 1x. This tests both the step and the scale-in.
- Pass criteria: zero 5xx responses from the load balancer, p95 latency under 300 ms, all 30 instances in service within six minutes of the 10:00 step, database connections below 80% of the new cap, Redis p95 below 5 ms, and no sustained CPU above 70% after stabilization.
- Warm-up evidence: new instances must pass readiness only after the JVM health endpoint is green; the load balancer must not send traffic earlier. Test that a deliberately slow-starting instance is removed rather than serving errors.
- Cloud Insight alarms: load balancer 5xx rate above 0.05% for two minutes, p95 latency above 300 ms for five minutes, unhealthy host count above zero, Auto Scaling desired versus in-service gap above two for five minutes, DB connections above 80%, and Redis evictions or latency spikes.
- Actions: page on-call for 5xx and unhealthy hosts; open a ticket for CPU and connection alerts; trigger a runbook step to add capacity manually only if the Auto Scaling group is at max. Alarm thresholds must be tuned in the two-week test, not on promotion day.
3. Finance asks for a hard monthly cost ceiling without risking the promotion. Which levers do you use?
Read the worked answer
- Cap the group at 30 instances; that is the technical cost ceiling. Anything above 30 would also break the database connection budget.
- Keep the baseline at ten rather than overprovisioning 30 all month. Use the 09:30 scheduled action and 12:15 scale-in so the extra 20 instances run for less than three hours.
- Right-size the instance type using the load test: if 30 instances at 4 vCPU are needed, verify that memory is not the constraint. Moving to a larger instance can reduce instance count but may worsen the connection-per-instance math.
- Set a budget alarm in billing at 80% and 100% of the monthly forecast, and review daily during the promotion week. Exclude non-production resources from the forecast and stop idle test servers after the load test.
- Do not use the cost ceiling as a reason to disable pre-scaling. A short, planned burst is cheaper than a failed promotion, and the scheduled action is bounded by the max size.