Choosing and Interpreting a Therapy Outcome Measure
An outpatient team is rolling out a 12-session cognitive-behavioral group for panic symptoms. They must choose a primary outcome for internal quality review. Options include: a 7-item panic symptom scale (score 0–28), a single 0–10 'how are you feeling' rating, and a behavioral measure counting panic attacks in the past two weeks. The team has 60 patients per year, collects data at baseline, mid-point, and end, and loses about 30% of patients to follow-up. Staff want a number that 'proves the group works.' One clinician notes that many patients improve on the single rating but not on the panic count. The coordinator asks which measure to use and what conclusions the data can support.
1. Compare the three candidate measures on the properties that matter for this quality review, and recommend a primary outcome.
Read the worked answer
The 7-item symptom scale offers several items, a defined range, and likely better reliability and sensitivity to change than a single item; it can be reported as a mean change with a confidence interval. The single 0–10 rating is quick but unreliable, coarse, and prone to response bias, so it is weak as a primary outcome though usable as a secondary global impression. The panic-attack count is concrete and clinically meaningful but highly variable, often zero-inflated, and sensitive to recall and definition problems over two weeks. Recommendation: use the 7-item scale as the primary outcome, with the panic count and a functional measure as secondary outcomes, and pre-specify the analysis before data collection. Report baseline and end scores, change scores, and the proportion meeting a meaningful improvement threshold, and track attendance and dropout.
2. With 30% attrition and no control group, what can and cannot be concluded if end-of-group scores improve?
Read the worked answer
Improvement cannot be attributed to the group. Without a comparison condition, maturation, regression to the mean, concurrent treatment, seasonal change, and repeated testing can all produce gains. Differential attrition is a further threat: patients who worsen may be the ones who leave, inflating apparent improvement. The team can conclude that completers reported lower scores, describe the size of change with confidence intervals, and compare completers with dropouts on baseline data. It cannot claim the group caused the change or generalize to all referrals. To strengthen inference, the team could add a waitlist or usual-care comparison, collect data on non-completers, use intention-to-treat principles where feasible, and pre-register the review plan.
3. The clinician observes that many patients improve on the single rating but not on the panic count. How should this discrepancy be interpreted?
Read the worked answer
Divergent results across measures are informative, not merely inconvenient. The single rating may capture general distress, hope, or therapeutic alliance rather than panic frequency, and it is more susceptible to demand effects. The panic count may be stable because the group reduces fear and avoidance without eliminating attacks, or because counting is unreliable. The team should examine correlations among measures, check whether change on the symptom scale tracks change on the count, and ask patients what changed. If the group mainly reduces avoidance and distress while attacks persist, that is a legitimate outcome to report, with the measures chosen to match the intended target. The response is to refine the outcome model and report multiple domains, not to select whichever number looks best.