Measurement and Data Collection
What this domain tests
Measurement and data collection tests operational definitions, continuous and discontinuous recording, accuracy and reliability, and interobserver agreement (IOA). Expect questions on choosing a measure that matches the behavior's dimensions and on interpreting IOA.
Core rules
- Operational definitions are observable, measurable, and complete; avoid internal states and vague labels.
- Continuous measures: frequency/rate, duration, latency, interresponse time, percent of opportunities.
- Discontinuous measures: whole-interval, partial-interval, and momentary time sampling. Whole interval tends to underestimate; partial interval tends to overestimate high-rate behavior.
- Permanent products can be measured without the behavior occurring.
- IOA methods: total count, mean count per interval, exact agreement, and block-by-block. Trial-by-trial is common for discrete responses.
- Acceptable IOA is often 80% or higher, with 90% preferred for high-stakes decisions.
- Observer drift, reactivity, and bias threaten accuracy.
Worked example
A teacher wants to measure on-task behavior during a 20-minute lesson. On-task is defined as eyes on materials and writing. Because it is a duration-like behavior, momentary time sampling every 2 minutes is practical and less reactive than continuous duration recording. Two observers collect MTS data on the same session and calculate exact agreement; agreement is 90%. Reasoning: the measure matches the dimension and the classroom constraints; IOA supports believability.
Traps
Using partial-interval recording for high-rate behavior and interpreting the overestimate as real; using frequency for behavior with long duration; reporting IOA from only easy sessions; failing to define the behavior before collecting data.