A 56-patient CBT trial, a borderline p-value, and a press release

Attrition and the analysis set. Differential attrition of 25% versus 7% is the largest threat. Completers-only analysis breaks randomization: if the seven CBT dropouts were the most symptomatic or least engaged, the remaining CBT group is a favorable subsample. Per-protocol analysis compounds this by removing the twelve low-attendance participants - precisely those for whom the treatment failed - and the jump from d = 0.53 to 0.79 quantifies the bias rather than resolving it. An intention-to-treat analysis with a principled approach to missing data is required before any efficacy claim.

Fragility and multiplicity. The primary result sits on the edge: a lower confidence bound of 0.2 and p = .046 mean a small amount of recalculation could reverse it. The quality-of-life finding is one of several uncorrected secondary comparisons, and the alliance measure was collected in only one arm, so it cannot be compared or used to explain outcome.

Design limits. A waitlist comparison controls for almost nothing: no attention, no expectancy, no therapist contact. Outcomes are self-report and participants are unblinded, so demand characteristics and allegiance effects are live. Effect sizes from waitlist trials systematically overstate performance against credible control.

Analysis. Report the primary outcome on the full randomized sample with a pre-specified missing-data model (multiple imputation under a plausible assumption, or a mixed model using all available observations), with the number analyzed, the estimand, and sensitivity analyses under different missing-not-at-random assumptions. Present the per-protocol result as a secondary, clearly labeled estimate. Report all outcomes, including null ones, with effect sizes and confidence intervals, and identify the one pre-specified primary outcome - the rest are exploratory and need multiplicity control or explicit labeling.

The quality-of-life sentence. "Improves quality of life" is not supported. A single secondary outcome at p = .03 in an unblinded, waitlist-controlled trial with 25% differential attrition is hypothesis-generating. A defensible formulation: "At post-treatment, the CBT arm showed a modest advantage on the primary social anxiety measure, with a confidence interval that included small effects, and a secondary quality-of-life difference of uncertain reliability; these findings require replication against an active comparator." State the attrition prominently. Also separate statistical significance from clinical importance - an 11-point LSAS difference may or may not cross the anchor-based threshold, and the trial should say which threshold it used in advance.

Design. Replace waitlist with an active comparator (for example, group supportive therapy or a credible attention control) so that nonspecific factors are equated. Use blinded assessors for the primary outcome and supplement self-report with a clinician-rated measure. Pre-register the primary outcome, the estimand, the analysis model, and the missing-data strategy. Track fidelity, therapist effects, adverse events, and dropout reasons. Collect alliance in both arms if alliance is to be modeled.

Sample size. Power the trial for the smallest difference that would change practice, not for the effect observed in the pilot. Anchor that difference to published minimal clinically important difference estimates for the LSAS - commonly in the range of roughly 10–12 points, though values vary by anchoring method and should be justified explicitly - and inflate for the anticipated attrition observed here. The prior trial's d = 0.53 is an upward-biased estimate from a small waitlist study, so planning on it would under-power the confirmatory trial; plan for a smaller difference and report the assumptions. Pre-specify the primary analysis as intention-to-treat and add a pre-registered subgroup hypothesis rather than a fishing expedition over attendance patterns.