AWS Solutions Architect: Where the 47 Minutes Went: Aurora Global Database Against a 15-Minute RTO

Where the 47 Minutes Went: Aurora Global Database Against a 15-Minute RTO

A ticketing platform runs in us-east-1 as primary with eu-west-1 as DR. Tier 1 (checkout and ticket issuance) must meet RTO 15 minutes and RPO 1 minute; tier 2 (reporting, search reindex) can tolerate RTO 4 hours and RPO 15 minutes. The database is Aurora MySQL with a Global Database secondary in eu-west-1, 2.4 TB, peaking around 1,800 write IOPS. The eu-west-1 stack is a pilot light: AMIs and launch templates exist, but zero instances run. Route 53 records have TTL 300 and a health check requiring three consecutive failures at 30-second intervals. In a game day, an unplanned regional failure restored tier 1 in 47 minutes. The postmortem notes engineers promoted the secondary manually after a 12-minute paging delay, then found the application read the database hostname from a Parameter Store value that still pointed at us-east-1.

1. Decompose the 47 minutes and state the changes required to reach a 15-minute RTO. Be explicit about what Aurora Global Database does and does not do for you.

Read the worked answer

Budget the 47 minutes roughly as: ~12 minutes paging and human acknowledgement; ~10 minutes deciding and executing promotion by hand; ~6–7 minutes of DNS staleness (TTL 300 s plus up to 90 s of health-check failures is 390 s worst case); the balance in cold-start of the pilot-light stack and the Parameter Store misconfiguration.

What Aurora Global Database gives you: asynchronous cross-region replication, typically sub-second lag under normal write volume, and a secondary you can promote. What it does not give you: promotion is not automatic for an unplanned regional loss, and promotion does nothing about DNS, application configuration, or compute capacity. Replication lag is your RPO risk, not your RTO risk - alarm on AuroraGlobalDBReplicationLag and treat any sustained lag above 60 s as an RPO breach, because 1 minute is the stated requirement and 'typically sub-second' is not a guarantee under a write burst.

Changes, in order of leverage:

Remove the human from the critical path. EventBridge health-check alarm -> Step Functions runbook that promotes the secondary, updates the hostname record, and scales the eu-west-1 ASG. Keep a human approval step only for a planned switchover.
Cut DNS staleness. TTL 30 s and a Route 53 Application Recovery Controller routing control, or a stable CNAME the runbook flips - not a Parameter Store value baked into a launch template.
Pre-warm the pilot light to a warm standby: minimum size 2 in eu-west-1 so the first request does not wait on an AMI boot plus userdata.
Rehearse quarterly and measure each segment. A 15-minute RTO with an unmeasured runbook is a hope.

If the runbook is still too slow, the honest answer is that tier 1 needs active-active reads with a local write path, not a faster failover script - but that is a much larger change than a 15-minute target justifies.

2. Tier 2 has a 4-hour RTO and 15-minute RPO. Justify the cheapest defensible pattern and the evidence you would require before signing off on it.

Read the worked answer

Backup-and-restore, not a second warm stack. Tier 2 is reporting and reindexing: it can be rebuilt from source systems if the restore is slow, and a second idle Aurora cluster plus a warm fleet would cost roughly the same as tier 1's DR footprint for a target that is 16x more forgiving. Take automated Aurora backups, copy them cross-region on a schedule, and keep the reporting stack as IaC that can be re-applied into eu-west-1.

The RPO is the binding constraint, not the RTO. Aurora continuous backup supports point-in-time restore to a granularity of about 5 minutes within the retention window, which satisfies 15 minutes - but only if the cross-region backup copy is recent enough to bound the loss. Copy continuously or at an interval well under 15 minutes, and alarm when the newest copy is older than 15 minutes. A nightly copy quietly turns your RPO into 24 hours.

Evidence before sign-off - three things, all testable:

Timed restore of the real 2.4 TB dataset. Restore time is not linear in size, and index warm-up after restore dominates the first queries. Time it, do not extrapolate it.
Service quota headroom in the DR account. A restore that fails because db.r6g.4xlarge is not approved in eu-west-1 is the single most common way these plans miss their target; request the quota now, not during an incident.
A documented reindex/replay procedure with a duration. If rebuilding search state takes 3 hours, restore plus rebuild must still fit in 4.

Reject this design if the business reclassifies tier 2, or if a quarterly restore test shows the elapsed time above ~3 hours - at that point the margin is gone and warm standby becomes the cheaper option once outage cost is counted.