Research Methods, Evaluation, and Informatics Scholarship
Core concepts
Understand study designs: randomized controlled trials, quasi-experimental (pre-post, interrupted time series), cohort, case-control, and qualitative methods. Know the difference between internal and external validity, and threats such as selection bias, confounding, and secular trends. Learn evaluation frameworks: the DeLone and McLean IS success model, and the distinction between formative and summative evaluation. Understand measurement validity and reliability, and the role of mixed methods. Know how to report informatics interventions (e.g., STARE-HI principles at a conceptual level) and the importance of reproducible methods.
Traps
Questions often treat a pre-post design as equivalent to an RCT. Another trap: ignoring secular trends that explain an improvement. Also watch for confusion between reliability (consistency) and validity (accuracy).
Rule of thumb
If you cannot rule out secular trends, use an interrupted time series or a control group. A single pre-post comparison is weak evidence.
Worked example
An alert reduces prescribing errors from 10% to 6% after launch. Without a control, the drop could reflect a concurrent policy change. An interrupted time series with a pre-period trend line strengthens the inference.