Eleven Days of Lead Time, Four Hours of Work
Lumen Outfitters is an e-commerce company with 9 product teams and 130 engineers running a Java storefront monolith plus 14 supporting services. Last quarter, deployment frequency was 2.4 per week and median lead time for changes (first commit to production) was 11.8 days. A value-stream exercise measured actual engineering plus automated build time on a typical change at about 4 hours. The remainder sits in queues: 5.1 days waiting for a slot in one of six shared QA environments (the 'ready for test' queue averages 210 changes), 2.7 days of release backlog because releases are cut every Tuesday, and 1.4 days waiting for manual regression sign-off from a 3-person QA team. Change failure rate is 19%, and QA spends 31% of its week re-testing behaviour already covered at service level. The VP of Engineering has asked for a plan to halve lead time in two quarters and has budget for one significant investment.
1. Using the numbers in the setup, locate the dominant constraint and explain why faster builds or more test automation will not by themselves halve lead time. Recommend the single investment and predict the effect.
Read the worked answer
Where the time actually goes
11.8 days is roughly 283 elapsed hours per change. Engineering plus build time is about 4 hours - around 1.4% of the total. Queueing accounts for 5.1 + 2.7 + 1.4 = 9.2 days, about 78%.
That ratio drives everything. A change waits roughly 70 times longer than it is worked on. Halving build time saves 2 hours out of 283 - about 0.7%, invisible in a median. Driving touch time to zero still leaves 11.7 days. So a plan that opens with faster CI caching is optimising the small term and will report a success that the metric cannot see.
Why the QA environment is binding
Six shared environments for 9 teams, with a 210-change queue, means utilisation is at or near saturation. Queue time grows non-linearly as utilisation approaches one, so the 5.1 days is a capacity property, not a discipline problem. The Tuesday train compounds it: a change merged on Wednesday waits up to six days regardless of readiness, so part of the 210-item queue is manufactured by the release cadence rather than by test capacity.
The single investment
Ephemeral, self-service environments created per change (preview environments on the existing cluster) attack the largest term and the batch size together. Removing the 5.1-day wait takes lead time to 6.7 days; making release on-demand rather than Tuesday-only takes it to about 4.0; an automated regression gate replacing manual sign-off gets to roughly 2.6. Halving is achievable without touching build speed.
Guardrail
Change failure rate is 19% and QA re-tests 31% of its week, so the plan must carry quality counter-metrics. Halving lead time by shipping more broken changes is not a win.
2. Design the measurement so that the claimed improvement cannot be produced by re-labelling or gaming, and name the counter-metrics you would publish alongside it.
Read the worked answer
Define the metric so it cannot be shortened by relabelling
- Unit of measurement: one change = one merged pull request, timed from the first commit on the branch to the moment that change's artifact digest serves production traffic.
- Report median and 85th percentile. A median alone hides the tail, and the 210-change queue lives entirely in the tail.
- Attribute deploys to artifact digest, not pipeline run ID. If a rebuild counts as a new deploy, the clock resets and the number improves without anything getting faster.
- Measure per change, not per release. A Tuesday release carrying 40 changes produces one deploy event; computing lead time from that event hides exactly the wait you are trying to remove.
- Include hotfixes and config-only changes. Excluding either is the cheapest way to fake a 30% improvement.
Counter-metrics to publish on the same dashboard
- Change failure rate and MTTR - prevents 'deploy more, break more'.
- Percentage of production changes requiring manual intervention - prevents declaring victory while humans still gate every release.
- Throughput of completed work items per month - prevents splitting one change into ten trivial commits to pull the median down.
- Rework rate (share of changes that are immediate fixes to a prior change) - catches half-finished work shipped to hit a number.
- QA rework percentage - should fall as the environment constraint is removed; if it rises, the new capacity is being used to test more broken changes.
Instrumentation audit
Check quarterly that deploy events come from the deployment system rather than a spreadsheet, and that the digest linkage actually resolves. If either check fails, the metric is fiction regardless of direction.