Forty-two slides, three attendees, no decisions
Arctos Logistics runs a monthly service review for its warehouse management and transport planning services. The pack has grown to 42 slides and 180 metrics, produced by two analysts over five days. It is presented by the service owner to an audience that has shrunk from eleven to three; the COO stopped attending eight months ago. Every metric is green, amber or red, against thresholds set in 2020 and never revised. Breaches are rare: four in the last twelve months, all of them availability. The continual improvement register holds three items, none of which has an owner, a baseline or a benefit estimate; two were closed as done without any measurement of effect. Separately, the transport planning team has complained for a year that its planning runs take too long, but no service level exists for that activity. An internal audit has just reported that service reporting does not support management decision-making, and the CIO has asked for a redesign before the next financial year.
1. Diagnose why this reporting and improvement cycle fails. Separate the measurement problems from the governance problems.
Read the worked answer
Measurement problems:
- Volume instead of relevance. 180 metrics exist because each team added its own; nobody removed any. A metric set that size cannot be read, so it is not read. The three remaining attendees are a symptom.
- Thresholds frozen in 2020. Targets set four years ago no longer reflect the service's demand, the business's expectations or what the technology can achieve. Green against a stale threshold is meaningless - and the fact that only availability ever breaches suggests the other thresholds are set where nothing can breach them.
- No customer-outcome measures. Everything reported is provider-side. The transport planning complaint has run for a year without appearing anywhere, because no measure exists for planning run duration. That is a measurement blind spot that has persisted for twelve months.
- No baselines. Without a baseline there is no way to say whether a change improved anything.
Governance problems:
- No decision rights. The review has no authority to commit resources, which is why the COO stopped attending. He was not wrong to stop; the meeting could not use him.
- One-way communication. The pack is presented, not discussed. There is no structured challenge and no customer voice.
- Orphaned improvement items. Three entries, no owners, no baselines, no benefits, two closed without verification. The register is a filing cabinet rather than a control.
- No linkage. Breaches are not connected to improvement actions, and improvement actions are not connected to the service levels.
- Production cost out of proportion. Ten analyst-days a month to produce something nobody reads is a resource decision that no one has taken.
The audit is right, and the fix is not a better pack. The failure is that reporting exists to demonstrate compliance rather than to support decisions. Adding more analysis would deepen the problem.
2. Design the replacement reporting approach: what to measure, at what level, for whom, and how often.
Read the worked answer
Rule one: design from the decisions backwards. For each audience, ask what decision they need to make and what they must see to make it. Anything that does not serve a decision is a candidate for removal.
Three layers:
- Strategic - quarterly, for the CIO, COO and the business service owners. Service performance against contractual commitments and against customer outcomes; the cost of the service and of the current level of quality; the top three risks; the improvement portfolio and realised benefits. Six to eight measures on one page. This is the meeting the COO should attend, and it should be about investment and risk, not uptime.
- Tactical - monthly, for service owners and product teams. SLO performance and trend, breaches and near misses, recurring incident themes, change failure rate, capacity and demand, and the open improvement actions. Fifteen to twenty measures with commentary on anything moving.
- Operational - continuous, self-service dashboards. Real-time and daily detail for the teams running the service. No one should be presenting this in a meeting; the meeting should assume it has been seen.
Selection criteria for every retained measure. It must be relevant to a stated objective, precisely defined, based on available data, timely enough to act on, agreed with the audience, and owned by a named individual. Apply this to the 180 and expect to retire most of them.
Add what is missing. A service level for transport planning run duration, defined with the planning team. A customer-experience measure - regular short surveys or operational feedback from warehouse and planning staff. And a benefit-realisation column in the improvement register.
Reset the thresholds. Rebaseline every retained metric on the last six to twelve months of data, agree targets with the business owners, and put a mandatory annual review on the calendar.
Automate the production. Target near-zero manual effort for the operational and tactical layers, and redeploy the analyst time to analysis rather than assembly. A monthly pack that takes five days cannot be produced often enough to be current and cannot be challenged with data on demand.
3. Apply the continual improvement model to the transport planning complaint, and explain how you would demonstrate that the improvement worked.
Read the worked answer
What is the vision? Planners can complete a planning run within the window needed to meet dispatch cut-offs, without manual rework, so that transport plans are current when the warehouse needs them.
Where are we now? There is no measure at all, so the first action is a baseline, not a fix. Instrument planning run duration and capture the distribution over four to six weeks: median, 90th and 99th percentile, volume per day, and the failure or rework rate. Interview three or four planners to find when the run is unbearably slow and what they do when it is. The complaint is a signal; the baseline turns it into a defined problem.
Where do we want to be? Derive a service level from the dispatch cut-off: for example, 95% of daily planning runs complete within the window, with a defined maximum. Agree it with the planning manager and record it as a service level requirement, then as an SLA commitment once there is confidence. Add an internal OLA with the data services team if the run depends on upstream data freshness - which is a common cause of slow runs.
How do we get there? Analyse the distribution for the causes: data volume growth, query performance, scheduling contention, upstream data arriving late. Prioritise the largest contributors and treat the rest as accepted for now.
Take action in small increments, and did we get there? - this is where most organisations stop, so make it explicit: re-measure the same distribution after each change, compare like with like, and use a control chart rather than before-and-after averages, because averages will flatter the result.
Demonstrating the benefit. Express it in business terms agreed in advance: planner hours returned per week, reduction in missed cut-offs, reduction in manual rework. Enter it in the improvement register with a baseline, a target, an owner, a measurement date and an agreed definition of the benefit, then report it at the quarterly strategic review. Also record the SLA added for planning runs as a durable outcome - the complaint existed for a year precisely because nothing converted it into a measurable commitment.