Appraising a Triage Before–After Study and a Resident's Ethical Request
A resident submits an abstract for a hospital research day. She reports that a new nurse-led triage protocol with standing orders reduced median door-to-doctor time in the emergency department from 45 minutes in 600 patients before the change to 31 minutes in 700 patients after it (p < 0.001), and concludes that the protocol 'reduced waiting time by 31%' and should be rolled out nationally. During the same period two additional physicians were hired, a second registration desk opened, and the winter bronchiolitis season began. The protocol was introduced personally by a popular department head. She also proposes adding two elements: an anonymous nurse satisfaction survey with a 100-shekel voucher for each respondent and oral consent only, and inclusion of data from a refugee-camp clinic that she does not intend to submit for ethics approval because 'it is just routine data'. Finally, she asks whether she can add a triage sepsis score with 90% sensitivity and 70% specificity, given a local sepsis prevalence at triage of about 5%.
1. Critique the internal validity of her before–after design and propose a stronger alternative.
Read the worked answer
The estimate is confounded and the causal claim is unsupported.
- Secular trend and co-interventions: two extra physicians and a second registration desk both reduce door-to-doctor time independently of the protocol. The result cannot be attributed to triage.
- Seasonality: bronchiolitis changes case-mix, arrival patterns and staffing, so the 'before' and 'after' periods are not exchangeable.
- Hawthorne and leadership effect: introduction by a popular department head may improve compliance or morale rather than the protocol itself working.
- Measurement: if outcome data were collected by the investigator who designed the protocol, ascertainment bias is possible; check whether door-to-doctor time was extracted from a time-stamped electronic system or recorded manually.
- Regression to the mean if the protocol was introduced after a particularly bad period.
Stronger designs, in order of feasibility: an interrupted time series with at least 12 monthly data points before and after, which allows level and slope change to be modelled; a stepped-wedge cluster design across several departments or hospitals; or, at minimum, concurrent controls in a comparable department with adjustment for staffing, season and case-mix. Report medians with interquartile ranges and the absolute difference with a confidence interval, not a percentage alone. A 14-minute absolute reduction is clinically meaningful - if it survives better design.
2. Interpret the p-value and effect size, and calculate what the sepsis score would mean at 5% prevalence.
Read the worked answer
The p-value. p < 0.001 means that a difference this large would be unlikely if there were truly no difference - it says nothing about size, causality or importance. With 1,300 patients, small and clinically trivial differences become 'statistically significant'. The 31% is a relative reduction; the absolute reduction of 14 minutes is what patients feel. Ask for the difference with a 95% confidence interval and whether it exceeds the smallest worthwhile effect.
Sepsis score at 5% prevalence, using a hypothetical 1,000 triage attendees:
- Diseased: 50. True positives 0.90 × 50 = 45; false negatives 5.
- Non-diseased: 950. False positives 0.30 × 950 = 285; true negatives 665.
- Positive predictive value = 45 / (45 + 285) = 13.6%. Negative predictive value = 665 / 670 = 99.3%.
So a positive score would be wrong about six times out of seven, while a negative score is reassuring. The score is therefore useful as a rule-out, not as a trigger for admission or antibiotics; using it as a rule-in test at this prevalence would cause large numbers of unnecessary treatments. Prevalence, not sensitivity alone, determines what a positive result means.
3. What ethical problems exist in her proposal, and what must happen before submission?
Read the worked answer
Several distinct problems:
- Consent. Oral consent only for a survey is acceptable in low-risk research if properly documented, but a 100-shekel voucher is a payment that could unduly influence staff who are in a dependent relationship with the investigator's department. The voucher should be modest, not contingent on completing the survey, and the voluntary nature must be explicit.
- Anonymity. A nurse survey in a single department is rarely truly anonymous; small cells and demographic details can identify respondents. Collect minimal identifiers and state how confidentiality will be protected.
- Refugee-camp data. 'Routine data' is not automatically exempt. Secondary use of identifiable clinical data requires ethics committee review, a data-protection plan, and, where practicable, consent - all the more so for a vulnerable population with limited ability to refuse. Labelling it routine and skipping review is a serious breach.
- Conflict of interest. The department head who introduced the protocol may be an author or supervisor; declare this.
- Approval. Nothing is submitted or presented until written approval from the hospital ethics committee is obtained, and data-sharing with the camp clinic is formalised.
Advise her early and constructively - this is a teaching moment, not a disciplinary one - and document the advice given.