The Recovery Target That Cannot Be Met
Meridian Foods runs a single order-management system that books 1,400 customer orders per hour during the 10:00–14:00 peak. Finance has declared a maximum tolerable outage of four hours. The vendor's replication option provides asynchronous replication with a measured average lag of 25 minutes and a worst observed lag of 90 minutes under load. A synchronous option exists but requires a second data center within 40 km; the only viable site is 310 km away, giving 9 ms round-trip latency that the vendor says will degrade peak throughput by roughly 30 percent. The CIO wants the synchronous option because it eliminates data loss. The operations director warns that a 30 percent throughput cut during peak will push order capture into backlog, which the business treats as an outage. The program has no approved budget increase and a 14-month deadline.
1. How should the program resolve the conflict between the CIO's data-loss concern and the operations director's throughput concern?
Read the worked answer
The conflict is not really about replication; it is about which failure the business is willing to accept. Frame the decision in the language of impact, not technology.
First, quantify what each option actually delivers. Asynchronous replication gives a recovery point objective somewhere between 25 and 90 minutes of lost transactions - at 1,400 orders per hour, that is roughly 580 to 2,100 orders that would need reconstruction from downstream records, plus whatever manual reconciliation that implies. Synchronous replication gives near-zero data loss but caps throughput, and the operations director's point is legitimate: sustained backlog at peak is an availability failure from the customer's perspective.
Second, test the assumption behind the CIO's position. Ask whether the order data can be reconstructed from another source within the four-hour recovery time objective. If order confirmations, payment authorizations, and warehouse pick tickets are retained elsewhere, the true data-loss exposure is smaller than the raw replication lag suggests, and the asynchronous option may be defensible.
Third, do not let this be settled by seniority. Present both options with their measured consequences to the business owner who owns the four-hour target, and record the accepted risk explicitly. If synchronous is chosen, the throughput degradation must be modeled against peak demand and a mitigation (load shaping, deferred batch work, or a second smaller site) identified before approval.
Finally, note the geographic constraint: a 310 km site also raises latency for any future synchronous workload, so the decision sets a precedent. Document the rationale so it is not relitigated at every review.
2. What should the program do about the 14-month deadline and the absence of budget increase?
Read the worked answer
Treat the deadline and budget as fixed constraints and design within them rather than assuming they will move.
A defensible path is phased delivery. Phase one implements asynchronous replication with a documented, tested recovery procedure and a reconciliation runbook that closes the gap between the 90-minute worst-case lag and the four-hour target. That meets the stated objective using existing budget. Phase two, if funded later, evaluates whether a nearer site or a different replication architecture can reduce data loss without the throughput penalty.
The key discipline is honesty about what phase one does and does not achieve. It meets the recovery time objective but leaves a recovery point exposure that the business owner must accept in writing. Presenting asynchronous replication as if it were equivalent to synchronous would be a misrepresentation that surfaces during the first real test.
Also validate the 14-month figure against dependencies: site readiness, vendor professional services, network circuits, and the exercise schedule needed to prove the capability. If the critical path shows the deadline is unachievable, say so early with evidence rather than discovering it at month twelve. An unrealistic plan approved quietly is worse than a scoped plan with a stated residual risk.