Recovery Site That Cannot Meet the Real Workload
A logistics company maintains a warm recovery site for its order-management system. The DR plan claims a 6-hour RTO. In a full failover test, the site restored the application in 5.5 hours - within target - but when 180 concurrent users reconnected, transaction response time degraded to 40+ seconds and the system could process only about 30% of normal order volume. The recovery site has 40% of production compute capacity and a network link sized for backup replication, not user traffic. The test report recorded 'RTO met' because the application was available. The business, however, defines recovery as resuming order intake at no less than 70% of normal throughput within 8 hours. The DR plan was written by IT and never reconciled with the business-defined recovery requirement. The next peak season begins in 90 days.
1. Was the RTO actually met? Analyze the gap between technical recovery and business recovery.
Read the worked answer
No - the RTO was not met in business terms.
- The test measured application availability, not service recovery. The business defines recovery as ≥70% of normal order throughput within 8 hours. The failover delivered ~30% throughput with 40-second response times - well below the threshold and arguably unusable for order intake.
- Root cause is capacity and bandwidth mismatch. 40% compute and a replication-sized link cannot carry the reconnected user load. The application being 'up' is necessary but not sufficient.
- Definitional disconnect. The DR plan's 6-hour RTO was an IT-authored figure never reconciled with the business-defined recovery requirement (70% throughput in 8 hours). Two different definitions were being measured against each other.
- The test report is misleading. Recording 'RTO met' based on availability alone overstates capability and would mislead management and auditors.
Conclusion: Technical recovery occurred; business recovery did not. The RTO should be judged against the business-defined throughput-and-time criterion.
2. With 90 days until peak season, what should be done, and how should the test and plan be corrected?
Read the worked answer
90-day remediation (prioritized):
- Reconcile definitions immediately. Rewrite the DR plan's recovery objective to match the business requirement: ≥70% throughput within 8 hours, with explicit throughput and response-time acceptance criteria.
- Size the recovery site to the requirement. Determine compute and network needed to sustain 70% throughput; upgrade the recovery site and/or link, or implement a staged ramp (prioritize order intake over reporting/batch).
- Interim mitigation for peak season. If full upgrade cannot land in 90 days, pre-arrange cloud burst capacity or a reduced-scope recovery that guarantees order intake throughput, and document the residual gap for management acceptance.
- Re-test against business criteria before peak, measuring throughput and response time at target concurrency, not just application availability.
Correct the test and plan:
- Add business-defined acceptance criteria (throughput, response time, concurrency) to every failover test.
- Include business representatives in test design and acceptance.
- Require IT and business to jointly own and sign the recovery objectives.
- Report test results as pass/fail against business criteria, with capacity headroom noted.