Redesigning Measurement When SLAs Are Met but Customers Complain
Horizon IT Services provides an e-commerce platform to 25 retail clients under SLAs. Its monthly service report states 99.95% availability and 92% of incidents closed within target, yet three clients have issued formal complaints about slow checkout during peak periods. Availability is calculated from server uptime, excluding planned maintenance and excluding failures of a third-party payment gateway. Incident closure time stops when the service desk closes the ticket; reopened incidents are logged as new tickets. Reports are compiled manually over 12 days after month end, and no one analyses trends. Supplier performance is reviewed only at annual contract renewal. The service manager says the SLAs are being met, so there is no need to change the measures.
1. Evaluate the current measurement approach against ISO/IEC 20000-1 clause 9.1. What does the standard require that Horizon is not doing?
Read the worked answer
What clause 9.1 requires. The organization must determine what to monitor and measure, the methods, when and by whom, and then analyse and evaluate the results. Measurement must cover service performance, the effectiveness of the SMS and processes, customer satisfaction, supplier performance, risks and trends, not just a narrow SLA calculation. Results must be retained as evidence and used to decide whether objectives are met.
What Horizon is missing. Availability is measured at component level and excludes a critical supplier, so it does not represent the customer's checkout experience. Incident closure stops at the service desk, and reopening creates a new ticket, which distorts the measure. Reporting is 12 days late and manual, so it cannot support timely decisions. There is no trend analysis, no supplier measurement and no evaluation of whether the service meets customer outcomes. The three complaints are evidence that the measure is not fit for purpose, not proof that the customer is wrong.
Judgement. Meeting an SLA is not the same as measuring service performance. A measure that excludes a dependency and rewards ticket closure rather than service restoration is a governance problem. Clause 9.1 would require Horizon to redefine the measures and demonstrate that they support evaluation and improvement.
2. Propose a redesigned set of measures for availability, incident management and supplier performance. Explain how each avoids the current distortions.
Read the worked answer
Availability. Measure end-to-end checkout success and latency at peak times using synthetic transactions, alongside infrastructure availability. Include third-party payment gateway failures in the customer-facing measure. If planned maintenance is excluded, report it separately and define the exclusion in the SLA rather than hiding it. Show the customer both the SLA calculation and the actual experience.
Incident management. Measure time to restore service, not time to close a ticket. Track the percentage of incidents reopened within seven days and report them against the original incident. Add first-time-fix rate, major incident review completion and backlog age. These measures remove the incentive to close early or split tickets.
Supplier performance. Set service measures for the payment gateway and other critical suppliers: transaction success rate, response and restore times, and the supplier's own incident reporting. Review them monthly with the supplier and feed them into service reviews. Annual contract renewal is too late.
Design rules. Each measure needs a definition, data source, owner, frequency and target. Use a small balanced set rather than a single index, and report within three working days after month end. Publish the calculation method so clients can see that the number is not being engineered.
3. The service manager argues that SLAs are met, so there is no need to change the measures. How should measurement results be used in management review and improvement when customers still complain?
Read the worked answer
SLA compliance is a floor, not the objective. Management review should consider customer complaints alongside SLA reports and ask whether the SLA definitions reflect the outcome customers need. If checkout is slow during peak periods while server uptime is high, the measure is incomplete or the threshold is wrong.
Triangulate inputs. Management review should examine end-to-end performance, complaint themes, incident and problem trends, supplier performance, risks and improvement actions. The three formal complaints are a signal, not an anomaly. The review should record a decision to investigate peak-load performance and the payment gateway dependency, assign an owner and set a date for evidence of improvement.
Use the improvement cycle. A corrective action under clause 10.2 addresses the immediate missed outcome; a continual improvement action under clause 10.1 redesigns the measures and the service. If the SLA itself is inadequate, the service relationship management process should propose a revision with the customer. The service manager's position should be challenged: an SMS that meets its own SLA while customers experience slow checkout is not demonstrating effective service management. Management review must show that it evaluated this evidence and acted.