Tidewater Logistics: Spot Capacity, Six-Hour Deadlines, and a 50% Cut
Tidewater Logistics scans and OCRs 2 million shipping documents a day. Each document takes about 40 seconds of CPU on an m5.large-class instance, and results are written to S3 keyed by document ID, so reprocessing a document is harmless. Work arrives unevenly, but the entire backlog must be cleared within six hours of receipt. The company currently runs 200 On-Demand instances around the clock, which at roughly $0.10 per instance-hour costs about $14,400 per month, and average CPU utilization is 35%. Finance has mandated a 50% reduction in compute spend within one quarter without missing the six-hour deadline, and engineering has refused any work that changes business logic. A previous Spot experiment failed when a single instance type in one Availability Zone was reclaimed all at once and the queue stalled for two hours.
1. Which purchasing options should be used for which part of this workload, and what saving would you expect?
Read the worked answer
The shape of the work decides the answer. Documents are idempotent and interruption-tolerant, which is the textbook definition of a Spot workload. The six-hour window is a throughput requirement, not a per-job latency requirement, so churn is acceptable provided capacity is replaced quickly.
Run the bulk of OCR on Spot with a diversified request across several instance families and all Availability Zones, using the capacity-optimized allocation strategy so requests land in the deepest pools. Keep a modest On-Demand baseline to drain the tail of the queue and hold the deadline when Spot capacity tightens, and cover that baseline with a one-year, no-upfront Compute Savings Plan rather than committing the whole fleet.
Rough arithmetic: the current $14,400 per month; Spot at a typical 60-70% discount puts the bulk near $4,300-$5,800, plus a 20-instance On-Demand baseline around $1,400, for roughly $6,000-$7,200. That clears the 50% target with headroom. Model it in Cost Explorer rather than trusting a headline discount, and note the conclusion changes if the deadline is ever tightened.
2. Explain why the earlier Spot experiment stalled and what changes prevent a recurrence.
Read the worked answer
The experiment treated Spot as one pool and discovered that AWS reclaims capacity per pool, not globally. A single instance type in a single Availability Zone is one pool; when that pool is reclaimed, every instance in it goes at once. Spot also delivers a two-minute interruption notice through instance metadata and EventBridge, and the pipeline apparently did nothing with it.
Fixes: diversify the request across at least three instance families and all Availability Zones, and use capacity-optimized allocation, which selects pools with the lowest interruption probability rather than the lowest price; handle the two-minute warning by checkpointing and exiting cleanly so the queue can redeliver; decouple with SQS so no dispatcher waits on a specific host; keep processing idempotent so redelivery cannot corrupt output; and alarm on queue depth and interruption rate.
The stalling symptom is the giveaway - the queue did not drain because nothing refilled the lost capacity. Spot is not unreliable; a single undiversified pool is.
3. Finance wants 50% off in one quarter without changing business logic. Is that realistic, and what else should be examined?
Read the worked answer
Fifty percent in a quarter is realistic here, but because the current baseline is wasteful rather than because of any single trick. Average CPU at 35% on 200 fixed instances suggests the fleet was sized for a peak that may not exist; right-sizing from CloudWatch metrics and AWS Compute Optimizer is the first move and often yields 20-30% alone.
Second, evaluate Graviton instances for the OCR workers, which typically offer better price-performance - but note that rebuilds the container image for arm64, so confirm whether engineering counts that as business logic. Third, lifecycle the OCR output in S3: results are read heavily for a week and rarely afterwards, so a transition to S3 Standard-IA or Glacier Instant Retrieval saves real money. Fourth, look for data transfer charges and forgotten development resources.
Finally, tag by workload and set AWS Budgets with Cost Anomaly Detection, or the savings will quietly erode. If the target still is not met, the next lever is reducing the 40 seconds per document - which is business logic and out of scope.