The 88% That Wasn't
Marcus, 9, engages in hand-mouthing that the team tracks with 10-second partial-interval recording during a 10-minute independent-work block. On Tuesday the RBT's sheet shows 27 intervals scored (45%). A second observer, seated across the room, scored 24. The two sheets agree on 22 intervals in which both recorded the behavior and on 31 intervals in which neither did. The RBT texts the BCBA: 'IOA was 88%, we're good.' Session percentages for the week read 45%, 62%, 45%. The BCBA had asked whether the new chewy-tube program is working, and the RBT wants to reply that it is, since Tuesday came in above Monday.
1. Verify the 88%. Show the arithmetic and explain what that number conceals.
Read the worked answer
A 10-minute block at 10-second intervals is 60 intervals (600 s ÷ 10 s).
- Agreements: 22 (both yes) + 31 (both no) = 53
- Disagreements: 5 where only the RBT scored + 2 where only the observer scored = 7
- Check: 53 + 7 = 60 ✓
Interval-by-interval IOA = 53/60 = 88.3% - the RBT's number is arithmetically right.
What it hides: most of that agreement is agreement that the behavior did not happen. Hand-mouthing occurred in under half the intervals, so 31 of the 53 agreements are 'both no.' The stricter scored-interval IOA is 22/(22+5+2) = 22/29 = 75.9%.
Both numbers should be reported. Even 88% sits below the usual 80–90% criterion the team uses, and 76% is well below it. The disagreement is not random noise either - the observer consistently scored fewer intervals, which suggests observer drift or a different working definition. Until the observers are retrained and rechecked, treat this week's data as unreliable.
2. What should the RBT tell the BCBA about whether the chewy-tube program is working?
Read the worked answer
The RBT should report, not conclude.
What she can state as fact:
- Three sessions, all run with pending IOA problems: 45%, 62%, 45%.
- No baseline phase is described, so there is nothing to compare against.
- Measurement agreement is below criterion, so even the direction of the numbers is uncertain.
What she should not say: 'It's working because Tuesday beat Monday.' One session above another is not an effect; the series is variable (up 17 points, then back down) and rests on data of doubtful reliability.
A defensible message: 'Here are the three sessions. IOA this week was 88% total and 76% scored-interval, so I'd like to recalibrate before we read a trend. Do you want me to keep the current measurement or change it?'
Interpreting graphed data and making treatment decisions is the BCBA's role; the RBT's job is to hand over accurate numbers and flag the measurement problem before a decision is made on them.
3. The BCBA suggests switching to duration-per-occurrence recording. Why does partial-interval inflate scores, and what does that mean for comparing 45%, 62%, 45% with future sessions?
Read the worked answer
Partial-interval scores an interval as 'occurred' if the behavior appears anywhere in it, even for one second. So it systematically overestimates how much of the time the behavior is happening, and it has a ceiling problem: a behavior occurring briefly but often can approach 100% while occupying little real time.
Consequences here:
- 45% is not 'hand-mouthing 45% of the block.' It is an index, useful for comparing sessions measured the same way, not for describing quantity.
- If the team switches to duration, the numbers will drop sharply for purely methodological reasons. Anyone reading the graph later could mistake that drop for improvement.
- So the switch has to be marked on the graph, ideally after a phase line, with a few sessions of IOA on the new method before the data are interpreted.
- Partial-interval also requires the observer to watch continuously and judge 'any occurrence,' which is exactly where two observers drift apart - another reason the 76% scored-interval figure matters.
The RBT should ask for the new definition and data sheet in writing rather than improvising a conversion.