Fifteen-Minute RPO on a 200 Mbps WAN
A financial services firm runs a 45 TB Oracle OLTP database on a Dell storage array at its primary site and replicates to a DR site 600 km away with 12 ms round-trip latency over a 200 Mbps IP WAN. Daily change is 400 GB, but 70% of it occurs in a four-hour end-of-day batch window; peak change rate is three times the daily average. The business requires an RPO of 15 minutes and an RTO of 2 hours. The current design uses storage-based asynchronous replication with a single consistency group. During last quarter's test, the DR database was 38 minutes behind at failover, and the application team needed 3 hours to reopen services because DNS and connection strings were manual. The network team says the link is 'only 20% utilized' based on a 24-hour average and refuses an upgrade.
1. Calculate the bandwidth required during the batch window and assess the network team's claim.
Read the worked answer
Daily average change is 400 GB / 86,400 s = 4.63 MB/s, or about 37 Mbps. Three times average is about 111 Mbps. But the average hides the batch: 70% of 400 GB is 280 GB in four hours, which is 70 GB/h, 19.4 MB/s, or about 155 Mbps. Add roughly 15% for replication and protocol overhead and the requirement is about 178 Mbps. On a 200 Mbps link that leaves only about 22 Mbps, or 11% headroom, before any production contention, retransmission, or burst. The network team's 20% utilization figure is a 24-hour average and is not a valid sizing basis for a four-hour batch peak. The link may technically carry the modeled batch, but it has insufficient margin for a 15-minute RPO during bursts, link flaps, or batch overrun. I would require QoS or traffic engineering to protect replication, and plan an upgrade to at least 300-400 Mbps or a second path. The RPO is a peak requirement, not an average requirement.
2. Diagnose the likely causes of the 38-minute lag and recommend changes to meet the 15-minute RPO.
Read the worked answer
The 38-minute lag at failover is consistent with a backlog created during the batch window, not with a steady-state link shortage. The batch pushes about 178 Mbps at peak against a 200 Mbps link, so any production contention or retransmission causes the asynchronous delta set to grow. If the consistency-group cycle or cache flush interval is longer than 15 minutes, the measured lag will exceed the RPO even when the link is healthy. Other causes include a single consistency group containing volumes with different write bursts, cache exhaustion, and replication sharing the link without QoS. Changes: protect replication with QoS or a dedicated path; reduce the batch concentration or spread it; tune the consistency-group cycle and cache so the RPO is measured and alerted; monitor delta-set size and drain time, not just link utilization; and confirm that the array's async replication can sustain the peak. If the array-based option cannot meet the cycle and multi-site requirements, evaluate RecoverPoint for consistency groups and bi-directional recovery, but do not change products to compensate for an undersized link.
3. What must change to bring the 2-hour RTO down from the observed 3 hours?
Read the worked answer
The extra hour is an orchestration problem, not a replication problem. The test shows manual DNS and connection-string changes, so the recovery runbook is serialized and dependent on people. I would automate DNS updates, connection strings, storage presentation, and database startup order, and define a single failover command with validation gates. Pre-stage application servers and network configuration at the DR site. Assign clear roles and a decision authority, and set a timed runbook with checkpoints: storage available, database mounted, application health check, and business validation. Then test to failure, not to a successful mount: measure from declaration to order processing, including DNS propagation and client reconnection. A 2-hour RTO for a 45 TB Oracle database is plausible only if the data is already replicated and the application layer is automated. I would also verify that the DR site has enough compute and I/O, because a fast storage failover followed by a slow database open still misses the RTO. Repeat the test quarterly and publish the measured result.