Demonstrating MTBF and Availability at 90% Confidence
A core network gateway must demonstrate MTBF >= 500 hours and availability >= 99.9% before a carrier will accept it. Reliability testing ran 20 units for 100 hours each under a uniform workload, 2,000 unit-hours in total, with 5 failures recorded: three software restarts caused by a memory leak in the session manager under long-lived sessions, one watchdog reset from a deadlock, and one hardware PSU failure. Mean repair time observed in the lab was 4 hours, including diagnostics and dispatch of a replacement unit. The product manager wants to sign the acceptance report now, arguing 'MTBF = 2,000 / 5 = 400, and with the leak fixed it will clearly exceed 500'. The customer's acceptance clause requires the demonstrated figure to be stated at 90% one-sided confidence. The carrier's traffic has a pronounced busy hour in which one signalling transaction type runs at about three times its daily average mix.
1. Compute the point estimate and the 90% one-sided lower confidence bound for MTBF, and state whether the acceptance criterion is met.
Read the worked answer
Point estimate: MTBF = total time / failures = 2,000 / 5 = 400 hours, already below the 500-hour requirement.
For a time-terminated test with a constant failure rate, the 90% one-sided lower confidence bound is:
MTBF_lower = 2T / chi-square(0.90, 2f + 2), with f = 5 failures, so 2f + 2 = 12 degrees of freedom, and chi-square(0.90, 12) = 18.549.
MTBF_lower = (2 x 2,000) / 18.549 = 4,000 / 18.549 = 215.6 hours.
So the team can state only that MTBF is at least about 216 hours at 90% confidence - less than half the requirement. The criterion is not met, and the point estimate does not meet it either.
What it would take: to demonstrate 500 hours at the same confidence with 5 failures, T must satisfy 2T / 18.549 >= 500, giving T >= 4,637 unit-hours, about 2,640 hours more than was run. If the memory-leak and deadlock fixes are verified and the failures attributable to them can be treated as a new test (a defensible position only with a demonstrated fix and a restart of the clock), the required additional time is lower but must be recomputed, not asserted. 'It will clearly exceed 500 once fixed' is a prediction; the acceptance clause asks for a demonstration.
2. Evaluate the availability claim and state what MTTR the 99.9% target requires.
Read the worked answer
Availability = MTBF / (MTBF + MTTR). Even granting the design target of MTBF = 500 h rather than the demonstrated 216 h:
- With MTTR = 4 h: A = 500 / 504 = 0.99206 = 99.21%, short of 99.9% by roughly 0.7 percentage points - about 69 hours of downtime per year against the 8.76 hours allowed.
- Required MTTR for 99.9% at MTBF = 500 h: MTTR <= MTBF x (1 - A) / A = 500 x 0.001 / 0.999 = 0.50 h (30 minutes).
At the demonstrated 216 h, the required MTTR would be 216 x 0.001 / 0.999 = 0.216 h, about 13 minutes - realistically unattainable for a swap-with-dispatch repair process.
The 4-hour lab MTTR includes diagnostics and dispatch; no field repair organisation will hit 30 minutes that way. The workable answer is architectural, not procedural: redundancy or failover that converts a repair into a sub-minute switchover, with the repair then happening out of service. That gives an effective MTTR in seconds. Note the trade-off honestly: failover changes what is being measured, introduces split-brain and failback risks, and must itself be tested with fault injection. Availability should be quoted for the redundant configuration, separately from the single-unit MTBF.
3. What validity problems does the 100-hour uniform-workload run have, and how would you redesign the reliability test?
Read the worked answer
Several issues undermine the inference from 2,000 unit-hours to field MTBF:
- Operational profile mismatch. Uniform workload does not represent the carrier's busy hour, where one signalling transaction runs at roughly 3x its average mix. Failure rates are workload-dependent; the test under-weights exactly the condition that will dominate field stress.
- Duration too short for the target. Each unit ran 100 hours against a 500-hour MTBF target - one fifth of the mean time between failures. Tests shorter than the target MTBF mostly sample the wrong part of the failure distribution and cannot expose slow degradation.
- The memory leak is duration-driven. A leak under long-lived sessions may need hundreds of hours to manifest. Ironically the leak was found, but the run is too short to bound its residual risk after the fix.
- Mixed failure classes. One PSU failure is hardware; the other four are software. They should be reported separately against separate targets, otherwise a hardware supplier's number and the software team's number are averaged into a figure that means nothing.
- 'Good as new' assumption. The model assumes repair restores the unit to its original failure rate. For a software defect fixed mid-test, the correct treatment is a new test segment starting at the fix, with the trend tracked.
Redesign: run a soak of at least 720 hours on a subset under a field operational profile including busy hour; run the full population to at least 3x the target MTBF cumulative; inject faults to validate failover MTTR; and plot a reliability growth curve (for example Duane or Crow-AMSAA) across fix segments rather than quoting one end-point average.