The 18-Minute Lead Time and the 11-Week Promise
FreshCart, an online grocery with 300 engineers and 40 deployable services, reports median commit-to-deploy lead time of 18 minutes, 60 deployments per day, change failure rate 2%, and MTTR 45 minutes. Its product director says a typical customer-visible change still takes 11 weeks from accepted idea to production, with 70% of the elapsed time waiting for prioritization or legal review. Engineering teams have begun splitting deployments into smaller batches to raise deployment frequency. Finance proposes a single 'value stream efficiency' KPI: story points delivered per developer per sprint. The VSM team is asked to present a measurement plan to the executive committee next month. The company has no consistent timestamp for when an idea becomes 'accepted demand,' and legal review uses a shared inbox. Two teams report lead time from different start points.
1. Explain the gap between the 18-minute lead time and the 11-week figure. What measurement defect must be fixed before they can be compared?
Read the worked answer
The two numbers measure different scopes. The 18-minute figure is DORA lead time for changes: it starts when a commit is made or a pull request is opened and ends when that change runs in production. The 11-week figure starts when an idea becomes accepted demand and ends when the customer sees it; it includes discovery, prioritization, legal review, scheduling, and release. Both are valid, but they are not comparable. The measurement defect is that 'accepted demand' has no consistent timestamp. Legal review sits in a shared inbox with no queue-entry event, so the 70% wait figure is an estimate. Before comparing, define the start event, for example a product owner records acceptance in the backlog with a timestamp, and the end event, customer-visible production use. Instrument stage transitions with the same work item ID across intake, legal, build and deploy. Until automation exists, reconstruct a sample of 15 recent items from emails and ticket dates, label confidence, and publish the method. Keep DORA as a delivery-subsystem metric rather than replacing it.
2. Recommend a metric set that reflects value stream health without incentivizing batch-splitting or individual productivity. Define each metric's start and end.
Read the worked answer
Use a small set of paired metrics. Value stream lead time: accepted demand timestamp to customer-visible production use; report p50 and p85, not the mean, and segment by work type. Flow efficiency: active touch time divided by elapsed time, calculated per stage and end to end, so long queues are visible. WIP by stage: count of items between each stage's entry and exit; high WIP predicts longer waits. Throughput: items completed per period at the value stream level. Queue time at the top three waits: legal, prioritization and environment, measured from entry to first active work. Rework rate: items that return to a previous stage. Keep the four DORA measures for the delivery subsystem, but pair deployment frequency with batch size and change failure rate so teams cannot inflate frequency by splitting deployments. Add one outcome metric such as feature adoption or order conversion. Do not use story points per developer; it rewards estimation inflation and local output. Review trends with teams, not individuals.
3. Finance wants one 'value stream efficiency' number. How should the VSM team respond?
Read the worked answer
Finance's proposed KPI measures local output, not value stream performance: story points per developer is easy to inflate, ignores wait time, and encourages work that is easy to estimate rather than valuable. A single composite index is also risky because it hides trade-offs: a team can improve lead time by dropping quality or by excluding regulatory work. The VSM team should offer one headline with guardrails. For example, use p85 value stream lead time for customer-visible changes as the headline, and report flow efficiency, change failure rate, and rework rate alongside it. Show the baseline, the distribution, and the segments, not a target that invites gaming. If the executive committee insists on one number, define it transparently, fix the weights for at least two quarters, and publish the component metrics so the number can be audited. Never attach the index to individual performance. Tie the discussion back to the customer promise, such as onboarding time or delivery reliability, because that is the outcome the value stream exists to produce.