Design Resilient Architectures
Core concepts
Resilience questions are framed by RPO (data loss tolerated) and RTO (downtime tolerated). Read those numbers first; they eliminate most options. Then reason in layers.
Compute and networking: spread across at least two AZs using an Auto Scaling group with subnets in each AZ, behind an ELB that health-checks and drains connections. A single NAT gateway is a classic single point of failure; one per AZ is the resilient answer. Elastic IPs and instance store are anti-patterns for durability (instance store is ephemeral).
Data: RDS Multi-AZ is for failover (synchronous standby, no read offload); read replicas are for read scaling and are asynchronous. Aurora offers replicas, backtrack, and Global Database for cross-region. S3 gives durability via redundancy plus versioning and cross-region replication. DynamoDB global tables give multi-region active-active.
Decoupling: SQS between tiers absorbs spikes; set a dead-letter queue for poison messages, and remember visibility timeout versus consumer processing time. Step Functions and EventBridge coordinate long workflows.
DR strategies from cheapest/slowest to most expensive/fastest: backup and restore, pilot light, warm standby, multi-site active-active.
Traps
- "Highly available" does not mean "Multi-AZ RDS increases read throughput."
- Deploying two instances in the same subnet or one AZ is not HA.
- SQS FIFO has lower throughput and does not support every feature standard queues do.
Worked example
A workload requires RTO of 15 minutes and RPO near zero across regions. Backups alone fail the RTO; multi-site active-active is overkill. Warm standby - a scaled-down but fully functional copy in a second region, continuous data replication, and Route 53 failover routing - fits.