Measurement, Data Analysis, and Reporting
Know the instrument before you interpret it
HCAHPS is a standardized, publicly reported survey of recently discharged adult inpatients, sampled randomly, fielded by mail, phone, or mixed modes, and adjusted for patient mix before public reporting. Its top-box for frequency items is “always,” not “usually.” Composite scores and star ratings are derived measures - a single item dropping does not automatically move a star. Vendor and mode differences make cross-organization comparisons unreliable unless the same instrument, mode, and timeframe are used. Every reported score should carry its sample size.
Analysis discipline
Distinguish common-cause variation from special-cause variation. On a run chart, six or more consecutive points on one side of the median (some texts use seven or eight) signals a shift; a single-point swing rarely does, especially with small n. Confidence intervals widen sharply with small samples, so low-volume units produce volatile scores that invite overreaction. Nonresponse bias matters: responders differ from nonresponders.
Mixing numbers and narrative
Quantitative surveys, real-time feedback, complaints and grievances, compliments, and verbatim comments answer different questions. Comments explain why a score moved; they do not replace the score. Benchmarking against percentile rank is not the same as benchmarking against the mean - top performers are a different comparison group.
Traps
Treating a small-sample change as real; comparing scores across vendors or survey modes; confusing top-box with mean score; inferring causation from correlation; cherry-picking favorable comments; ignoring response rate and nonresponse bias; reporting a percentage without its denominator.
Rule of thumb
Before reacting to any change, ask three questions: what is n, is this special cause on the chart, and is the comparison group equivalent? If any answer is unclear, the correct exam answer is usually “gather more data,” not “act now.”