Program Evaluation, Quality, and Outcomes
Program evaluation judges whether a program is effective, efficient, and appropriate. Know the distinction between structure, process, and outcome measures (Donabedian) and between quality improvement (iterative, local) and research (generalizable, hypothesis-testing). Understand PDSA cycles, root cause analysis, and the Model for Improvement (aims, measures, changes).
Know common quality frameworks and metrics: HEDIS, CMS Star Ratings, readmission and ED-visit rates, patient experience (HCAHPS), and benchmarking. Understand outcome measures (clinical endpoints, utilization, cost, quality of life) versus process measures (screening rates, adherence to protocols) and balancing measures (unintended consequences). Recognize validity, reliability, sensitivity, and specificity when selecting measures.
Traps: confusing a process measure with an outcome measure; attributing improvement to a program without a comparison (regression to the mean, secular trends); using a single metric as proof of success; ignoring balancing measures; and confusing QI (no IRB approval needed for standard QI, but publication may require review) with research. Another trap is choosing an easy-to-measure metric that does not reflect what matters to patients.
Rule of thumb: A credible evaluation names the measure type (structure/process/outcome), the comparison, and the timeframe - and includes balancing measures to catch unintended harm.
Worked example: A care transitions program reports fewer readmissions. A strong evaluation compares to a control period or group, adjusts for case mix, tracks process measures (follow-up calls completed), and monitors a balancing measure (ED visits) to ensure readmissions were not merely shifted.