Autoscaling a One-Hour Morning Peak
A SaaS provider runs eight fixed Public Cloud instances behind a load balancer. CPU sits at about 20% overnight but reaches 85% every weekday from 09:00 to 10:00, when users log in and generate reports. The team wants to add two larger instances, but finance has asked for a 30% reduction in off-peak compute spend. User sessions are stored in local memory on each instance, and generated PDFs are written to each instance's local disk. The managed database is 4 vCores and 16 GB RAM and reaches 70% CPU during the peak. The load balancer uses round-robin without health checks beyond TCP. The application can run multiple copies but is not yet stateless. A three-week release window is available. The team also has a nightly batch job that runs on one instance and must not run more than once.
1. Why won't larger instances meet the cost goal, and what application changes are prerequisites for autoscaling?
Read the worked answer
Larger fixed instances would still run at low utilisation for 23 hours a day, so they increase cost rather than remove it. The one-hour peak needs elasticity, but autoscaling will fail until the application is stateless. Move sessions to a managed Redis or a shared database table; move generated PDFs to Object Storage and serve them through signed URLs; add a /health endpoint that checks dependencies; make shutdown graceful and requests idempotent. Then put the instances in an autoscaling group behind the load balancer, minimum two across availability zones, maximum twelve, with CPU 60% and request-count policies and a five-minute cool-down. Switch the load balancer to HTTP health checks. Run the nightly batch as a singleton outside the group, or protect it with a distributed lock. Load-test at 10x before changing production.
2. Design the database and availability-zone topology so the database does not become the next bottleneck.
Read the worked answer
The database is already at 70% CPU at peak, so it becomes the bottleneck as soon as application capacity increases. First pool connections and cache read-heavy queries; then resize the managed database vertically and add a read replica for reporting. Put the database in a 3-AZ region with a standby if the service offers it, and verify the failover process. Keep application instances spread across zones, and ensure the load balancer balances across zones. Do not autoscale the database. Set backups with point-in-time recovery and test a restore. Watch slow queries and replication lag; if the report job is the cause, move it off-peak or to the replica. The nightly batch should connect to the primary only if it writes.
3. What quota, cost and failure guardrails would you put in place before enabling autoscaling?
Read the worked answer
Before the release, request quota increases for compute, volumes and public IPs in the target region; autoscaling silently fails when quota is exhausted. Set a maximum of twelve instances and a budget alert at the expected monthly cost. Use conservative scale-in and a cool-down to avoid thrash, and pre-warm images so new instances start within the peak window. Add dashboards for request latency, 5xx rate, queue depth and database CPU. Test instance failure and zone failure. Review autoscaling logs weekly. Keep a manual override to pin capacity during a campaign, and document the cost of the maximum footprint, not just the average. Confirm the batch job cannot be duplicated by a scale-out event.