When the BIA and the Infrastructure Team Disagree on RTO
Vantage Precision Components runs three plants on a single ERP instance (42 TB, 1,100 named users). The business owner signed a BIA declaring an RTO of 2 hours and an RPO of zero for ERP because "orders stop." Infrastructure's tested reality is 11 hours to restore from backup, 4 hours with the existing warm standby, and an estimated $2.4M in capital plus $380k per year to reach a genuine 2-hour capability. Separately, IT set a 72-hour RTO for the plant CNC scheduling system without consulting operations, yet a scheduling outage idles 600 workers at roughly $95k per shift-hour of lost output. The only Oracle DBA who maintains the ERP runbooks retires in 90 days, and no one else has performed a production database restore. The steering committee wants a decision by Friday.
1. Which BIA values are wrong, and what analysis produces defensible numbers for both systems?
Read the worked answer
Both figures are unsupported, but for opposite reasons
ERP at 2 hours is a business-stated aspiration, not a derived requirement. "Orders stop" is a symptom, not an impact curve. The BIA must quantify impact over time - revenue deferred versus lost, contractual penalties, regulatory exposure, safety - and identify the point at which impact becomes unacceptable. That point is the MTD. The RTO is then set so that RTO + WRT ≤ MTD. If the business can take manual/EDI orders for 12 hours and batch them later, the MTD is not 2 hours, and the $2.4M capability buys insurance against a risk that may already be mitigated by a cheap workaround.
CNC scheduling at 72 hours is a technology guess with no business input - a classic IT-driven RTO. At $95k per shift-hour of idle output, 72 hours of downtime is roughly $6.8M of lost output, which almost certainly exceeds the MTD. This system, not ERP, may be the organization's real Tier-1 exposure, and it was invisible to the committee.
The method to defend on Friday
- Re-interview functional owners with a time-based impact questionnaire (1h, 4h, 8h, 24h, 72h), quantifying each dimension.
- Map dependencies: ERP depends on identity, database, interfaces, and network - the RTO of the whole chain is the slowest component, not the fastest.
- Set cost-of-downtime against cost-of-capability and pick the crossing point; that is the economically rational RTO, and it should be documented as a business decision with an owner.
Do not present the committee with "the business is wrong." Present them with an impact curve and a price for each point on it.
2. The business refuses to reduce the 2-hour ERP RTO. What recovery design and governance response does that justify?
Read the worked answer
Stop treating ERP as a single indivisible object
A 2-hour RTO for 42 TB of ERP is expensive because it is being bought for the whole system. Almost no business genuinely needs every module in 2 hours. Decompose by function and assign different targets:
- Order capture and shipping/EDI interfaces - the actual revenue-stopping functions. These are small data sets and can run on a lightweight standby or a read/write buffer with synchronous replication. A 2-hour target for this slice is affordable.
- Finance, MRP, reporting, historical data - restore from backup in 12–24 hours; nobody closes the books during a failover.
This tiered design typically delivers the business outcome at a fraction of $2.4M, and it is honest: the full ERP is not back in 2 hours, but the process that generates revenue is.
Governance response
If the business still demands the full-system 2-hour RTO, do not silently accept an undeliverable commitment. Present three options - fund $2.4M, accept tiered recovery at lower cost, or accept the exposure - with cost, residual risk, and the downtime consequence of each. Record the chosen option in a risk acceptance register signed by the business owner. That signature is the deliverable; the DR plan must never contain an RTO that the tested capability cannot meet, because the gap will surface in the first real invocation and destroy confidence in the whole plan.
Finally, add an RPO caveat: "RPO zero" implies synchronous replication and zero data loss under all failure modes. Confirm the business understands the cost of that, or restate it as a few minutes of asynchronous lag.
3. What must be done about the retiring DBA before Friday's decision, and why is it a DR issue rather than an HR one?
Read the worked answer
The single point of failure is a person, not a server
A recovery capability that only one individual can execute is not a capability - it is a hope. If the DBA is unavailable during an actual invocation, the tested 11-hour restore time is meaningless; it becomes whatever an untrained team can improvise.
Actions, in priority order
- Prove the runbook. Have a second engineer perform a full production-equivalent database restore using only the written runbook, with the DBA observing silently. Every place the second engineer must ask a question is a documentation defect. Repeat until the restore is completed unaided.
- Capture undocumented knowledge - storage layout, RMAN/backup configuration, license keys, support contract numbers, vendor escalation paths, and the quirks that never made it into the document.
- Contract a backstop before the retirement date, not after: vendor premium support or a managed service partner who can be engaged during an invocation. Note that a backstop who has never touched the environment is only marginally better than no backstop - include them in the next test.
- Add DR competence to the risk register. Key-person dependency across all Tier-1 systems should be assessed as part of the BIA, not discovered during a resignation.
Why it belongs in Friday's decision
The committee is being asked to spend $2.4M on technology resilience while an unfunded, near-certain knowledge-loss event sits 90 days out. Present both together: the DBA risk is cheap to mitigate and would invalidate the expensive option if ignored.