Research Methods and Measurement
What this domain covers
Research Methods and Measurement tests how educational claims are produced and evaluated. Expect items on variables, designs, validity threats, descriptive and inferential statistics, reliability, validity, and research ethics.
Core rules
- Variables: the independent variable is manipulated; the dependent variable is measured. Operational definitions specify how a construct is measured.
- Designs: experimental designs manipulate a variable and use random assignment to support causal inference. Quasi-experimental designs lack random assignment. Correlational designs describe relationships but cannot establish causation. A pretest-posttest design without a control group is vulnerable to maturation and testing effects.
- Sampling: random sampling supports generalization; random assignment supports group equivalence. They are not the same.
- Validity threats: history, maturation, testing, instrumentation, regression to the mean, selection, and attrition threaten internal validity; artificial settings threaten external validity. Internal validity is confidence in a causal claim; external validity is generalizability.
- Statistics: mean, median, and mode describe central tendency; standard deviation describes spread. Correlation coefficients range from -1 to +1. Descriptive statistics summarize the sample; inferential statistics estimate whether findings generalize. A p-value is not the probability that the null hypothesis is true, and statistical significance is not effect size. Effect size, such as Cohen's d, shows magnitude.
- Measurement: reliability types include test-retest, inter-rater, and internal consistency. Validity types include content, construct, and criterion. The standard error of measurement estimates error around an observed score. Informed consent, confidentiality, and IRB review are standard ethical requirements.
Worked example
A researcher finds that students who read more have higher grades and concludes that reading more causes higher grades. The flaw: this is correlational. A third variable such as prior achievement or family support could influence both, and students choose how much they read. A stronger causal test would randomly assign students to a reading intervention or control and compare outcomes, with fidelity checks.
Traps
Correlation does not imply causation. Random assignment is not random sampling. A statistically significant result can be trivial in size. Reliability does not guarantee validity. Regression to the mean can look like an intervention effect when groups are selected for extreme scores. Attrition can bias results if dropouts differ from completers.