APCAT study materials
APCAT study resources

Fidelity Falls Off a Cliff at Week Five

APCAT · Put your knowledge to work in context.

All case studies

Fidelity Falls Off a Cliff at Week Five

You are leading a 90-day rollout of a new intake protocol across four teams (28 staff) at a community services organization. Weeks 1–3 go well: fidelity checklist scores average 88% across 24 observed intakes. In week 4 the senior clinician who championed the protocol resigns. By week 5 fidelity has fallen to 54%, and three team leads report staff are completing the checklist "from memory" at the end of shift. Two staff say the form takes 12 minutes per intake and they see an average of 5 intakes a day. No one has been directly observed on the two risk-screening items since week 3. The board reviews progress at day 60.

1. What is your reading of the drop, and what do you verify first?

Read the worked answer

A 34-point drop in one week is almost certainly not a motivation problem - and it is not yet a measured fact either.

  • Check the measurement first. Weeks 1–3 numbers came from 24 directly observed intakes. In week 5 staff are self-reporting from memory. You are comparing observation to self-report, which are different instruments. Get a fresh round of direct observation before diagnosing anything, starting with the two risk-screening items - they have not been observed since week 3 and are the only items with immediate safety consequences.
  • Check the mechanism. The champion's departure almost certainly removed the informal prompts and immediate feedback that were doing the real work. The written protocol never carried them. Fidelity decayed when the discriminative stimuli left the building.
  • Check the workload math, because it is real. 12 minutes × 5 intakes = 60 minutes per person per day on documentation, on top of the intake itself. That is a plausible competing contingency, not an excuse.

Distinguish skill deficit (cannot), prompt deficit (forgot), and consequence deficit (no reason to) before choosing a fix.

2. Decide what to change: modify the protocol, retrain, or both - and state the criteria you will hold.

Read the worked answer

Both, in a fixed order, because they address different failures.

  1. Fix measurement this week. The checklist must be completable at the point of contact, not reconstructed at end of shift. Move it into the record system so completion is timestamped, and spot-check five intakes per team per week.
  2. Engineering fix by day 35. The 12-minute form is the problem. Cut to core items, target ≤4 minutes, and make the two risk-screening items non-skippable. Do not cut risk items to buy time - that trades a documentation problem for a safety problem.
  3. Restore the feedback loop. Designate one peer coach per team with a 10-minute weekly observation and immediate feedback. This replaces what the champion was doing informally and does not depend on any single person.
  4. Targeted retraining only on the items actually failing in observation. Re-running the full 90-day training spends the credibility you need later and fixes nothing.

Criteria I will hold: two consecutive weeks of ≥85% observed fidelity, with risk items at 100%. If risk items are not at 100% within two weeks, this escalates as a client-safety issue rather than a quality-improvement one.

3. What do you present to the board at day 60?

Read the worked answer

Report the state of the implementation, not the state you wish it were in.

  • Where we are: weeks 1–3 at 88% observed fidelity across 24 observations; weeks 4–5 at 54% but on a changed measurement method; current status pending re-observation on a stated date. Do not present 88% as the current number.
  • What caused it: loss of an informal coaching role, a documentation burden of 60 staff-minutes per day, and self-report substituting for observation. None of these are attitude problems, and naming that matters for how the board responds to staff.
  • Risk disclosure: the two screening items went unobserved for two weeks. State the interim mitigation - an audit of all intakes from week 3 forward, with 100% of flagged cases reviewed.
  • What we need from the board: a decision on the shortened form, approval for peer-coach time (roughly 2 h per coach per week), and either a timeline extension or explicit acceptance of a narrower scope.
  • What we will report next: a single defined metric - observed fidelity with interobserver checks - at day 90.