The Customer Satisfaction Score That Went Up While Customers Left
Harbourline is a business-to-business equipment servicer with 4,200 contracted customers. Its balanced scorecard reports a customer satisfaction (CSAT) score of 91%, up from 84% two years ago, and the board treats this as evidence of improving service. Meanwhile: contract renewals have fallen from 88% to 79%; average response time to breakdowns has risen from 4.1 hours to 6.8 hours; first-time-fix rate has fallen from 82% to 71%; and engineer headcount has been cut by 12% while the contract base grew 9%. The CSAT survey is sent only to customers who have logged a call that was closed as 'resolved', and is completed by 6% of recipients. The operations director says the score is 'the only independent voice we have'. A large customer has just issued a service-credit claim.
1. Critically assess the reliability of the CSAT measure as a performance indicator in this setting.
Read the worked answer
The measure is not fit for purpose. Key defects:
- Selection bias. Only customers whose call was closed as 'resolved' are surveyed. Customers with unresolved failures - precisely the ones likely to churn - are excluded by construction. The 79% renewal rate is direct evidence of the excluded population.
- Non-response bias. A 6% response rate means the score represents roughly 250 customers out of 4,200. Respondents are likely the satisfied minority; there is no weighting or non-response analysis.
- Construct validity. CSAT measures satisfaction with a single interaction, not overall relationship health. It is a lagging, attitudinal measure and is weakly linked to the outcome the board cares about (retention and revenue).
- Gaming and drift. The definition of 'resolved' is set internally and can drift; a falling first-time-fix rate (82%→71%) suggests more repeat visits, yet calls can still be closed.
- Contradiction with objective data. Renewals, response time and first-time-fix all deteriorated. When a perception measure moves opposite to hard operational data, the measure, not the operations, should be suspected.
Conclusion: the operations director's claim that it is 'independent' is wrong - it is internally filtered. It should not be a primary KPI and certainly not a bonus trigger.
2. Recommend a redesigned customer-perspective measurement set, justifying each measure and its linkage to financial outcomes.
Read the worked answer
Design principle: pair a leading operational driver with a lagging outcome, and use objective data wherever possible.
| Measure | Type | Why |
|---|---|---|
| Contract renewal rate (target ≥88%) | Lagging outcome | Directly drives revenue; the board's real objective |
| Net revenue retention | Lagging outcome | Captures expansion/contraction, not just logos retained |
| Response time to breakdown (target ≤4 hrs) | Leading driver | Objective, hard to game, causally linked to renewal |
| First-time-fix rate (target ≥82%) | Leading driver | Reduces repeat visits and cost-to-serve |
| Service-credit value as % of revenue | Lagging quality proxy | Financial consequence of failure, self-policing |
| Relationship survey (all customers, annual, census not sample) | Attitudinal | Independent of call closure; measures overall relationship |
| Complaint escalation rate | Leading | Early-warning of churn risk |
Justification of the causal chain: faster response and higher first-time-fix → fewer repeat calls and credits → higher perceived reliability → higher renewal and net revenue retention → revenue and margin. The scorecard should show this chain explicitly so the board can see whether the leading indicators predict the lagging ones.
Survey fixes: survey the full customer base (or a random stratified sample) regardless of call status; report response rate alongside the score; weight by contract value; add a 'would you renew?' intention question as a leading churn indicator. Retire standalone CSAT as a bonus metric until these controls exist.