Reducing a Noisy Batch Platform's Cost Without Breaking SLAs
A media company runs a nightly ETL fleet on 40 m5.4xlarge EC2 instances behind an Auto Scaling group, plus 12 always-on m5.2xlarge instances for a metadata API. Batch jobs take 6 hours, are CPU-bound, and are latency-tolerant (must finish by 06:00). Monthly compute spend is ~$38,000. CloudWatch shows batch CPU averaging 22% with spikes to 90%; the API instances average 35% CPU but have p99 latency of 180 ms against a 250 ms SLO. The team has no Savings Plans. A prior attempt to use Spot for batch caused job failures when capacity was reclaimed mid-run, and the team reverted.
1. Propose a cost plan that distinguishes the batch fleet from the API fleet, and explain the commitment instruments you would buy and why.
Read the worked answer
Treat the two fleets differently because their risk profiles differ. API fleet (always-on, latency-sensitive): buy a Compute Savings Plan (1-year, no upfront or partial upfront) covering the steady 12-instance baseline. Savings Plans apply across instance family and Region, so a future resize from m5 to m6i does not strand the commitment. Do not use Spot here - p99 180 ms against a 250 ms SLO leaves too little headroom for interruption-driven capacity churn, and the fleet is small relative to batch.
Batch fleet (latency-tolerant, deadline-bound): use a mixed instances policy with Spot as the base and On-Demand as the diversified fallback, spanning multiple instance families (m5, m6i, m5a, c5) and AZs. The earlier failure was a single-instance-type Spot request with no diversification and no checkpointing. Fix both: checkpoint job state to S3 every N minutes so a reclaimed instance resumes rather than restarts, and set the ASG to rebalance and replace automatically. Because CPU averages 22%, the fleet is over-provisioned - right-size to m5.2xlarge or use a target-tracking policy on CPU with a floor, which may cut instance count before any purchase. Buy a smaller Savings Plan for the predictable On-Demand portion of batch. Validate with a canary week: run 10% of jobs on Spot and compare completion-by-06:00 rate before committing.
2. How would you prove the changes are safe before cutting over, and what metrics would you watch after?
Read the worked answer
Before cutover, run a shadow week: route a subset of jobs (say 10–15%) through the new mixed-instance ASG with checkpointing enabled, and compare job completion time, retry count, and 06:00 deadline attainment against the current fleet. Inject an interruption deliberately (terminate an instance mid-job) to confirm resume-from-checkpoint works and that the job does not restart from zero. For the API fleet, run the Savings Plan change in report-only mode first (Cost Explorer shows what the plan would have covered) and load-test at p99 to confirm no instance-type change alters latency.
After cutover, watch: batch job success rate and deadline attainment per night, Spot interruption rate per instance family (CloudWatch/EventBridge interruption notices), ASG scaling activity, and effective cost per completed job (not just hourly spend). For the API, watch p99 latency and 5xx rate against the SLO, plus Savings Plan utilization and coverage in Cost Explorer - target >90% utilization so you are not over-committed. Set a budget alert at 10% over the new baseline so a regression is caught within a billing cycle.