Reading a 120-patient malaria trial for a formulary decision

The p-value p = 0.012 means that if there were truly no difference in mean parasite clearance time between the two drugs, the chance of seeing a difference at least as large as 6.6 hours is 1.2%. It is a measure of compatibility between the data and the null hypothesis. It is not the probability that the null hypothesis is true, and it is not the probability that the finding will replicate.

The confidence interval The 95% CI of 2.9 to 10.3 hours means that if the trial were repeated many times, 95% of intervals constructed this way would contain the true difference. Because the interval excludes zero, the result is statistically significant at the 5% level, which is consistent with p = 0.012. The interval also carries the clinically useful information that the benefit could be as small as about three hours or as large as about ten, so the estimate is imprecise - 60 patients per arm is a small study.

Statistical versus clinical significance A mean difference of 6.6 hours in parasite clearance is real but modest, and parasite clearance is a surrogate. What patients and clinicians care about is death, coma, need for transfusion and length of stay. The mortality difference, 3/60 versus 5/60, is not statistically significant and the trial had no power to detect a clinically important mortality difference - such a trial would need thousands of patients. So the paper supports a biological advantage for artesunate but cannot demonstrate the outcome that matters most. A further concern is whether parasite clearance was measured blind, since unblinded reading of blood films can bias a soft endpoint.

The multiplicity problem If five independent outcomes are each tested at alpha = 0.05 when all null hypotheses are true, the probability of at least one false positive is 1 - 0.95⁵, which is about 23%. So one 'significant' secondary result out of five is roughly what chance alone would produce. The hypoglycaemia finding may well be noise, and promoting it to the abstract is a form of spin that readers and journalists rarely correct for.

How it should have been handled

• Pre-specify a hierarchy in the protocol: one primary outcome, a small number of key secondary outcomes, and the rest explicitly exploratory.
• Control the family-wise error rate, for example Bonferroni (0.05/5 = 0.01 per secondary outcome) or step-down Holm; alternatively use false discovery rate methods such as Benjamini-Hochberg if screening many outcomes.
• Report every outcome with effect sizes and confidence intervals, not p-values alone, and never redefine the primary outcome after looking at the data.
• Label unadjusted secondary analyses as hypothesis-generating.

What to do with it Artemisinin derivatives do act quickly, so hypoglycaemia is biologically plausible and worth a properly powered trial. But it should not be used to justify a formulary change, and it should not appear in the abstract without a clear statement that it was an unadjusted secondary analysis among several.

Do not decide on this trial alone A 120-patient trial with a surrogate primary outcome cannot carry a formulary decision by itself. The committee should place it in the context of the whole evidence base. The larger body of trial evidence and WHO guidance already position intravenous or intramuscular artesunate as the preferred treatment for severe malaria, with intramuscular artemether or intravenous quinine as alternatives when artesunate is unavailable. So a switch is consistent with international recommendations - but that is because of the wider evidence, not because 6.6 hours reached p = 0.012.

Questions to ask the paper and the pharmacy

• External validity: two teaching hospitals, adults only. Children under five carry most malaria deaths, and the pharmacokinetics and formulation needs differ.
• Practicality: artesunate is given intravenously and needs reconstitution and a functioning drip; artemether is given by intramuscular injection, which matters in a rural ward where intravenous access in a convulsing child is hard.
• Cost and budget impact: an extra ₦2,700 per course; at 400 severe cases a year that is about ₦1.08 million extra. Ask for a cost-effectiveness estimate - cost per additional death or disability averted - rather than cost per course.
• Supply security, cold chain, staff training and the risk of stock-outs of the newer product.
• Safety data, including delayed haemolysis after artesunate, which requires follow-up haemoglobin checks.

Recommendation Adopt artesunate as first-line in line with national and WHO guidance, with artemether retained for patients in whom intravenous access cannot be achieved or during stock-outs, and build in audit of mortality, stock levels and follow-up haemoglobin. Do not present the decision to staff as being based on this single trial, and write the minutes to say so.