Autoscaling Web Tier Behind SLB During Flash Sale
A retail platform runs an ECS-based web tier behind a public-facing SLB (layer-4 TCP listener, round-robin). During a flash sale, request volume triples for 10 minutes. The scaling group is configured with min=4, max=20, a scaling rule that adds 2 instances when average CPU > 70% for 5 minutes, and a cooldown of 300 seconds. After the sale, operators report 502 errors for roughly 4 minutes at the start of the spike and again during scale-in. Instances boot a custom image that takes 210 seconds to register with the SLB health check after launch. The health check is TCP on port 80 with a 5-second interval, 3 healthy threshold, 2 unhealthy threshold.
1. Diagnose why 502 errors appear at the start of the traffic spike and propose a configuration change set that addresses the root cause.
Read the worked answer
The 502s at spike onset are a capacity/latency mismatch, not an SLB fault:
- Detection lag: CPU>70% for 5 minutes plus a 300s cooldown means the first scale-out cannot begin until ~5 minutes after load rises. The 10-minute spike is largely over before new capacity is healthy.
- Warm-up lag: new instances need ~210s to pass health checks, so even a triggered scale-out adds capacity too late.
- Cooldown interaction: a 300s cooldown after each scaling activity prevents a second wave of scale-out during a sustained spike.
Recommended changes:
- Lower the trigger window (e.g., 1–2 minutes) and/or scale on a leading indicator (SLB QPS/active connections via CloudMonitor) rather than CPU alone.
- Enable scale-in/scale-out with warm-up so new instances are not added to SLB until healthy, and use scheduled scaling for known flash-sale windows to pre-provision capacity before the event.
- Reduce cooldown or use target-tracking style policies that continuously adjust; ensure max is high enough (20 may be insufficient - validate with per-instance capacity math).
- Pre-bake the image or use a startup script so health-check readiness is faster; keep health check thresholds tuned so a slow-booting instance is not marked healthy prematurely.
- Confirm the SLB backend has enough weight/connection headroom and that the health check port matches the actual listener.
2. During scale-in, 502s recur. Explain the likely cause and how to make scale-in safe.
Read the worked answer
Scale-in removes instances from the scaling group. If an instance is removed from the SLB backend set and terminated while it still has in-flight connections (TCP listener, keep-alive), clients see resets/502s.
Mitigations:
- Use connection draining on the SLB so existing connections are allowed to complete before the backend is detached.
- In the scaling group, enable graceful removal / remove-from-SLB-before-terminate behavior so the instance is first taken out of rotation, then terminated after draining.
- For TCP listeners, ensure idle timeout and keep-alive settings are shorter than the drain window so connections do not linger indefinitely.
- Prefer a scale-in policy that removes the newest/least-loaded instances and respects a minimum healthy count; avoid aggressive scale-in immediately after a spike.
- Verify with SLB access logs and ECS system logs that terminations correlate with the 502 window.