Eight Months In: Did the Single-Family NICU Rooms Work?
Eight months after opening a new 32-bed NICU with all single-family rooms, hospital leadership asks your EBD team: 'Did the private rooms improve outcomes?' The data you can get: a parent satisfaction survey returned by 22% of families, versus 61% response in the old unit; nurse turnover at 19% against 11% system-wide; average LOS down 2 days; and breastfeeding at discharge up 8 percentage points. Meanwhile the unit moved to 12-hour shifts, launched a nurse residency program, and adopted a new feeding protocol. You have one part-time analyst for three months and no research budget.
1. Identify the validity threats and state which of the four findings you can and cannot attribute to the design.
Read the worked answer
Sort the findings by how much causal weight they can bear.
- Satisfaction (22% vs. 61% response): the least trustworthy. A collapse in response rate means the respondents are a different, self-selected group; the comparison is between two non-comparable samples, not two units. Treat as hypothesis-generating only.
- Nurse turnover (19% vs. 11% system-wide): not attributable. The comparator is other units, not the prior NICU, and three concurrent changes (12-hour shifts, residency, feeding protocol) plus a national nursing market confound it completely. History and co-intervention.
- LOS down 2 days: plausible in direction, but maturation and secular trend are live - NICUs nationally have been shortening stays - and the feeding protocol directly targets LOS.
- Breastfeeding up 8 points: the feeding protocol is the obvious competing explanation; single-family rooms plausibly help via maternal presence, but the protocol dominates.
Overall: this is a pre-post design with several simultaneous interventions and no control. You can describe change; you cannot attribute it.
2. Design a feasible evaluation with one part-time analyst and three months. What do you measure, and how do you handle the survey?
Read the worked answer
Narrow the question to what is answerable.
- Fix the survey first. Re-field with a mode change (text/QR at bedside plus a call-back for non-responders) and report response rate alongside results. If you cannot reach 45–50%, report with explicit non-response caveats and do not trend against the old unit.
- Build a comparison, not a before/after. Use a sister NICU without single-family rooms as a concurrent comparator for LOS and breastfeeding; a difference-in-differences with two time points is crude but far better than a single-unit trend.
- Use process measures as mechanism checks. Maternal presence hours, skin-to-skin minutes, expressed milk initiation, and staff time-in-room are closer to the design than LOS and move faster; they tell you whether the rooms are being used as intended.
- Segment. Compare inborn vs. outborn, gestational age bands, and room occupancy - a design effect should be strongest where exposure is greatest.
- Interview 8–10 nurses about workflow and turnover intent; cheap, and explains the number rather than just reporting it.
Timeline: weeks 1–2 instrument and data pull, weeks 3–8 field, weeks 9–12 analysis.
3. Write the headline you would give leadership, and explain how you resist the pressure to say the rooms 'worked.'
Read the worked answer
Headline: 'Outcomes moved in the expected direction on all four measures, but three other major changes occurred at the same time; we can say the unit is performing better, not that the rooms caused it.'
How to hold that line:
- Lead with what leadership can act on: the rooms are occupied, staff report X, and the mechanism measures (maternal presence, time-in-room) either support or fail to support the intended design hypothesis. That is decision-useful even without attribution.
- Show the counterfactual explicitly - a small table of the four outcomes with the alternative explanation named for each. It pre-empts the single-cause story.
- Offer the next step: a 24-month re-measure with the comparator unit, which will be far more credible once the launch period washes out.
- Do not overclaim in either direction. Saying 'no evidence it worked' when the direction is consistent and the mechanism measures are positive is as inaccurate as claiming proof.
Credibility is the asset; spend it on precision, not on a verdict leadership wants.