A Baseline Built on Three Definitions of Handle Time
TeleServe's average handle time (AHT) is 412 seconds in the ACD system, and a Black Belt is chartered to cut 40 seconds from it. Reviewing 30 recorded calls reveals: the ACD clock excludes after-call work for the outsourced vendor, which handles 35% of volume; QA counts a transfer as a new call; two supervisors coach to different definitions of handle time; and a 30-call attribute agreement study among three reviewers yields 71% agreement where chance agreement is about 45%. Baseline data covers weekdays 08:00–16:00 only. A three-week pilot on one 12-person team measured 374 seconds.
1. What must be fixed before 412 seconds can be accepted as a baseline, and in what order?
Read the worked answer
Order matters, because each step invalidates the next if skipped.
1. Operational definition. Write the start and stop points: does the clock include hold, after-call work, transfers, warm transfers, and system downtime? Is a transferred call one contact or two? Resolve with QA, operations, and the vendor together - this is a decision, not a discovery. Publish it as the single definition used in coaching, dashboards, and the charter.
2. Measurement system. Agreement of 71% against 45% chance is weak: kappa ≈ (0.71 − 0.45)/(1 − 0.45) ≈ 0.47, far below the 0.7 typically required for a usable attribute system and nowhere near 0.9 for critical classification. If AHT is measured continuously by system timestamp, run a Gage R&R on the measurement path (clock start/stop logic, clock-stopping behavior) rather than on human raters. Fix definitions and retrain, then re-run agreement. If agreement does not reach ~0.9, do not proceed.
3. Sampling frame. The baseline omits the vendor - 35% of volume - and excludes evenings and weekends. Whatever the vendor's definitional issue, excluding a third of the population guarantees a baseline that cannot represent the process.
4. Only then compute the baseline, stratified by vendor/in-house, shift, and call type, with the uncertainty stated.
2. Is the 38-second pilot reduction real? What specifically do you check before claiming it?
Read the worked answer
374 vs 412 is a 9% improvement on a 12-person team over three weeks - plausible but not yet evidence.
Check, in this order:
- Is the comparison valid? The 412 baseline includes the vendor and all shifts; the pilot is one in-house team on weekdays. You may be comparing two different populations. Recompute a like-for-like baseline for that team and shift before any subtraction.
- Call mix. AHT is heavily driven by call type and complexity. If the pilot weeks happened to carry simpler calls, the drop is mix, not performance. Compare call-type proportions or adjust with a regression controlling for mix, transfers, and tenure.
- Confounders and history. Three weeks is short. New-coaching effects (Hawthorne), a seasonal dip, or a supervisor change can produce a 38-second move with no process change at all.
- Statistical signal. Build an individuals chart of weekly team AHT with limits from a stable pre-pilot period. Ask whether 374 falls outside those limits and stays outside for consecutive weeks - not whether the point estimate looks smaller.
- Replication. Run the change on a second team with a matched comparison team left untouched. A finding that does not replicate on a second team is a hypothesis.
Claim 'promising, pending replication' and keep the charter target open.
3. The sponsor has already announced '412 seconds' at a town hall. How do you handle the baseline correction?
Read the worked answer
Correct it, but control the framing and the timing - the sponsor owns the correction, not you.
- Give the sponsor a private briefing first, with the number and the reason: the vendor's clock excludes after-call work and 35% of volume was outside the sample, so 412 is not a like-for-like figure. Bring a proposed replacement, not a problem.
- State the baseline as a range with its composition: in-house weekday AHT of X seconds (measurement uncertainty stated), vendor AHT measured on a separate clock, evenings and weekends included in the sampling plan going forward.
- Restate the goal in relative terms where possible ('reduce in-house AHT by 8% on a comparable basis') so the project is not hostage to a single number that will move when the definition is fixed.
- Sequence it: fix the definition and MSA, recalculate, then announce once. A second correction three weeks later is worse than one slightly uncomfortable one now.
- Log the change in the charter revision history so auditors and finance can see why the baseline moved.