Research
The Research domain tests whether you can choose a defensible design, judge data quality, and use findings to inform decisions. Items contrast formative and summative research, probability and nonprobability samples, and qualitative and quantitative methods.
Core rules
- Formative research precedes planning: audits, interviews, focus groups, baseline surveys. Summative research evaluates outcomes after implementation.
- Primary vs. secondary: primary data are collected for the current problem; secondary data already exist, so check currency, source, and method.
- Probability sampling supports generalization and margin-of-error estimates; nonprobability methods such as convenience, quota, and volunteer panels do not.
- Margin of error reflects sampling error only, not question wording, nonresponse, or coverage error.
- Validity means the instrument measures what it claims; reliability means consistency across repetitions.
- Pretesting messages and instruments reduces avoidable error; human-subject research may require review.
Worked example
A team wants to claim that 70% of Americans support the policy based on 500 responses to a volunteer online poll promoted on its own social channels. The poll is a nonprobability, self-selected sample; supporters are more likely to respond, and the frame excludes non-followers. You cannot generalize to Americans or attach a meaningful margin of error. The fix is a probability-based survey with a defined frame, random selection, and weighting - or report the finding only as among respondents to the online poll, with limitations stated.
Traps
- Attaching a margin of error to a convenience sample.
- Confusing correlation with causation.
- Writing leading or double-barreled questions.
- Treating impressions or web traffic as research findings rather than outputs.