Headroom, Hot Racks and a Generator That Would Not Start

Assess power, cooling and redundancy separately, because the binding constraint differs.

Power. 1.8 MW on 2.0 MW usable gives 200 kW of headroom, and that 2.0 MW figure assumes one module is already out for maintenance. The new request is 4 × 25 kW = 100 kW, which would take load to 1.9 MW - inside the N+1 envelope but with only 5% margin and no allowance for inrush, growth on existing racks, or a module being unavailable during a maintenance window that coincides with high load. The UPS load per module also matters: modules loaded above roughly 80% lose efficiency and ride-through margin.

Cooling. 2.2 MW installed with N+1 gives 1.65 MW usable against 1.8 MW actual. The cooling plant already cannot sustain a single CRAH failure at current load. This is the binding constraint, not power. The 27°C at the top of three racks at 22 kW is the physical evidence of insufficient local airflow, not a room-level capacity number.

Conclusion. You cannot commit to the four 25 kW racks on the current cooling configuration, and you should not treat 200 kW of power headroom as usable capacity. Commit only to what survives a single failure: at present, nothing above current load.

Answer Sales in writing. State the constraint, the remediation required (local containment or in-row cooling for high-density positions, additional CRAH capacity, and a rack-level power and thermal validation), and an earliest date. A verbal "has capacity" becomes a contract term in the customer's mind and a service credit in yours.

Treat the three failures as separate fault trees.

1. Generator 2 failed to start. Interrogate the start sequence: battery/starting system, fuel supply, controller fault log, and the last successful test date. Check whether the automatic test runs on no-load only - a no-load test can pass for years while the set cannot accept load. Verify starting batteries with a load test, not a voltage check.
2. Generator 1 above 80% within 12 seconds. This is a sizing and sequencing problem, not a fault. With one set out, the remaining set picks up the whole hall. Either block load by staged transfer, shed non-critical load on transfer, or accept that N+1 generation is not actually available. Determine which and document it; the current configuration means a single generator failure during an outage is a full loss of the hall.
3. Fuel autonomy. 6 hours on the day tank against a 48-hour commitment is a contract breach waiting for a long outage. Confirm bulk tank usable volume (not nominal), consumption at actual load, refuelling contract response time and the testing of the fuel polishing regime - 14 months without polishing risks microbial contamination and filter blockage under load, which typically shows up exactly when the generator is needed.

Re-test properly. Annual test should include: full-load bank test on each generator individually, a test with one generator isolated to prove the N+1 claim, staged transfer with real IT load, timed fuel consumption measurement, and a start test from cold with the site in its normal state. Record the sequence, not just pass/fail, and have the client witness it. A test that transfers load and declares success without isolating a generator has never actually tested resilience.

Prioritise by consequence and by what undermines a commitment already made.

1. Generator start reliability and fuel autonomy (0–90 days). Lowest cost, highest immediate risk reduction, and directly addresses a written 48-hour commitment that the site cannot currently meet. Includes starting-system overhaul, fuel polishing and testing, bulk volume verification, refuelling agreement with a stated response time, and a documented interim statement to clients on actual autonomy until the contract position is fixed.
2. Loading and transfer strategy (0–6 months). Quantify the cost of load shedding or staged transfer versus adding generator capacity. If the client contract promises N+1 generation, adding a set or reducing the connected load is not optional.
3. Cooling capacity and high-density airflow (3–9 months). The 1.65 MW usable figure against 1.8 MW actual is a live redundancy gap. Options: additional CRAH capacity, containment and local in-row or rear-door cooling for the 22 kW positions, and a rack-level thermal policy that caps inlet temperature with a defined alarm threshold. Fund containment and airflow first where it defers capital CRAH spend; model the efficiency effect on PUE as part of the case.
4. Rack density policy and provisioning gate (6–12 months). No rack above an agreed kW threshold without a validated power and thermal design, and a per-rack capacity register that Sales must check before quoting.

Business case structure. For each item: the commitment or resilience claim currently at risk, the failure scenario and its consequence (customer impact, credit exposure, safety), the cost, and the residual risk if deferred. Quantify availability exposure per incident rather than presenting percentages of improvement, and state explicitly that the 100 kW tenant request cannot be accepted until item 3 is complete. Investment committees fund avoided incidents more readily than they fund elegance; show them the incident.