Sizing for a Monthly Spike: One Big Instance or Many Small Ones

Reasoning

• Peak-to-baseline ratio is 320 / 40 = 8x. Whatever the absolute size, the fleet must handle eight times steady-state demand for roughly six hours a month.
• 12% average utilization on 32 vCPU means the current fleet is not sized for its steady load; nobody knows the real per-request cost. The first step is to measure vCPU-seconds per request under the current build and size to ~50% utilization at baseline. That lands baseline near 8-10 vCPU and peak near 64-80 vCPU.
• So the 64-vCPU box is about right for the peak and roughly 8x oversized for the other 29 days. You would pay peak capacity for 720 hours to use it for six.
• The 'cheaper per vCPU' argument optimizes unit price, not total cost. Larger instances carry a lower hourly rate per vCPU but a much higher hourly total.

Design problems with the single instance

• Host or zone failure is a total outage; there is no second copy.
• Every change requires a stop/start, so deployments become outages.
• Vertical scaling has a hard ceiling and no graceful degradation.

Recommendation

Replace with an autoscaling group of smaller instances across at least two availability zones: minimum two (one per zone), target tracking on requests-per-target or CPU at 50%, and a maximum sized to peak plus 30% headroom. Nothing in the application resists this: it is stateless, the session cache is already external, and report jobs are idempotent, so scale-out has no correctness cost. The saving comes from not paying for the peak around the clock, not from the per-vCPU discount.

• Reactive scaling has two lags: the metric delay (one or more datapoints plus a cooldown) and the four-minute instance warm-up. A fast ramp can outrun that even with aggressive policies.
• Use scheduled scaling to raise the minimum capacity about 30 minutes before the historical ramp, then let target tracking handle variation above that floor. The spike is predictable to the day and hour; treat it as a schedule, not a surprise.
• Shrink the warm-up: bake the JVM and cache priming into a pre-warmed image or AMI, or keep a small pool of paused workers that join the target group in seconds.
• Protect the floor from capacity risk: reserve capacity for the scheduled minimum and do not let the month-end window run on spot or on a single zone.
• Decouple instantaneous load from work: put report requests on a queue (the jobs are idempotent) so the system can absorb a burst and drain at a rate the fleet can serve. Cost then tracks throughput rather than the tallest minute.
• Prove it before month-end with a load test at 1.2x peak and record time-to-capacity.

Measure at the ramp and at baseline

• p95 and p99 latency in both windows, not averages.
• Scale-out events with time-to-capacity: how many instances launched, how long until healthy, and whether the queue depth grew during the ramp.
• Utilization at p95 rather than mean, so headroom is real.
• Cost per 1,000 requests, split by baseline month and month-end day, to confirm that the saving is genuine and not shifted into the spike day.

Revert or re-tune if any of these hold

• p95 latency breaches the SLO during the ramp, or queue depth grows monotonically for more than a few minutes.
• The autoscaler repeatedly hits its maximum, meaning the ceiling is wrong rather than the design.
• Cost per 1,000 requests rises rather than falls, which would mean the sizing math was wrong.
• A single day's spike consumes a disproportionate share of the monthly bill, in which case revisit scheduled rather than dynamic scaling or a queue-based buffering design.