Cloud spend up 38% but reliability down
A logistics company runs 180 microservices on a PaaS. Over 18 months, deployment lead time fell from two days to 40 minutes and release frequency rose from monthly to twice weekly. At the same time, change failure rate is 22%, mean time to restore is 4 hours 20 minutes, and cloud spend is 38% above budget. The finance team sees only a monthly invoice; no one tracks cost per shipment or per tenant. Monitoring generates 1,200 alerts a day, and engineers ignore most of them. Six product teams own services, but no one owns the end-to-end order-to-dispatch value stream. The CIO wants a three-month improvement programme with measurable outcomes and no hiring. Customer complaints about late tracking updates have doubled.
1. Define the baseline and the value stream before proposing tools. What would you measure, and what would you avoid optimizing first?
Read the worked answer
Establish a baseline for flow, reliability and unit cost: deployment lead time, change failure rate, mean time to restore, alert-to-action ratio, cost per shipment and per tenant, and late-tracking complaints. Map the order-to-dispatch value stream across the six product teams to find where work waits and where handoffs fail. Do not start by optimizing deployment frequency or buying another monitoring tool. Deployment lead time has already improved dramatically, yet change failure rate and restoration time worsened, which suggests local optimization: teams are deploying faster without sufficient testing, observability or end-to-end ownership. The 1,200 daily alerts are a symptom of poor signal design, not a reason to add more alerts. Cost is also invisible because the invoice is not allocated to shipments or tenants, so Finance cannot distinguish growth from waste. Measurement should pair speed with stability and cost so that one metric cannot be improved by damaging another. This is the starting point for continual improvement: understand the current state and the value stream before choosing countermeasures.
2. Build a three-month improvement plan using the continual improvement model. What are the first two iterations?
Read the worked answer
Vision: reliable, cost-transparent order-to-dispatch service without new headcount. Current state: baseline the metrics above and map the value stream. Target state: change failure rate below 10%, mean time to restore below 1 hour, alert volume reduced by 70% with no loss of detection, and cost per shipment reduced by 10% against baseline. First iteration: reduce alert noise and establish end-to-end ownership. Instrument the top customer journeys, retire duplicate alerts, route actionable alerts to an owner, and assign a value-stream owner who can arbitrate across the six teams. Second iteration: make cost and failure visible. Tag cloud resources to shipment or tenant where possible, publish a weekly unit-cost and reliability review, introduce blameless post-incident reviews and require runbooks for the top 20 services. Take action in two-week increments, check results against the baseline, and integrate successful changes into normal service management. Do not attempt a big-bang reorganization in three months. The sequencing follows progress iteratively with feedback and collaborate and promote visibility: fix signal and ownership first, because cost and reliability improvements depend on someone being accountable for the end-to-end outcome.
3. Propose a measurement set with counter-metrics that prevents gaming. How would you report it to the CIO?
Read the worked answer
Use a balanced set: deployment lead time, release frequency, change failure rate, mean time to restore, alert-to-action ratio, cost per shipment, and late-tracking complaints. Add counter-metrics deliberately: pair release frequency with change failure rate so speed cannot be bought with instability; pair cost per shipment with reliability and complaint rate so cost cannot be cut by degrading service; pair alert reduction with detection coverage and incident escape rate so noise reduction does not become blindness. Report trends against the baseline, not absolute targets alone, and show the value-stream view rather than six separate team scorecards. The CIO should see one page: customer outcome, reliability, flow and unit cost, with the counter-metrics beside each. Use measurement and reporting to support decisions, not to rank teams. The reasoning is that any single metric can be gamed locally; pairing it with a counter-metric makes the trade-off visible. Review monthly, adjust targets if the counter-metrics deteriorate, and keep the reporting cadence short enough to drive the next iteration.