ECS Auto Scaling Under Bursty Traffic
Your web tier runs on 12 ECS instances behind an SLB. CPU averages 35% but spikes to 90% for 3-4 minutes every hour when a batch job hits the API. Auto Scaling is configured with a target-tracking rule at 60% CPU and a 300-second cooldown. During spikes, response latency climbs to 4 seconds and some requests time out before new instances finish their 90-second boot and SLB health-check warmup. The ops team proposes lowering the target to 40% CPU. Finance notes the fleet already runs at 35% average and worries about cost.
1. Should you lower the target-tracking threshold to 40%? Justify your decision and propose a better scaling approach.
Read the worked answer
Lowering the target to 40% is the wrong lever. Target-tracking reacts to sustained CPU; a 3-4 minute spike is shorter than the 300s cooldown plus 90s boot, so the fleet would scale on the wrong signal and then pay for idle capacity the other 56 minutes of each hour. Instead:
- Switch to step or scheduled scaling keyed to the batch job's known schedule, pre-warming N instances 5 minutes before each run.
- Shorten cooldown or use lifecycle hooks to ensure new instances pass health checks before SLB routes traffic.
- Consider reserved/elastic mix: keep a small reserved baseline and use elastic instances for the predictable burst.
The 40% target would raise steady-state cost without fixing the tail latency, because the bottleneck is reaction time, not average headroom.
2. The batch job's schedule shifts by up to 20 minutes daily. How does this change your recommendation?
Read the worked answer
A fixed scheduled-scaling rule becomes unreliable when the trigger drifts. Options:
- Widen the scheduled window with a conservative pre-warm (e.g., start 25 minutes early) and accept some idle minutes - cheap relative to timeouts.
- Event-driven scaling: have the batch orchestrator publish a signal (e.g., to a message queue or via an API call) that triggers scale-out before the job begins, rather than relying on clock time.
- Keep a modest step-scaling rule as a safety net for unexpected spikes.
Event-driven pre-warm is the most robust because it scales on the actual cause, not a proxy. Validate with a load test that the warmup time (90s boot + health check) fits inside the pre-warm lead time.