Velocity up 38%, everything else down
A quarterly business review opens with the Product Owner's chart: velocity rose from 42 to 58 points per Sprint, a 38% improvement. Meanwhile, monthly customer-reported defects rose from 9 to 21; median time from an item being accepted into the backlog to production release is still 14 weeks; release cadence is unchanged at one per month; NPS fell from 31 to 27; and the share of backlog spend on new features fell as defect work grew. Two teams of different sizes appear on the same chart, compared with each other. The VP concludes the teams "have never been more productive" and asks for velocity to rise another 20% next quarter. The Scrum Masters are asked to respond.
1. Assess the VP's conclusion. What does the evidence actually support?
Read the worked answer
What velocity is and is not. Velocity is a planning aid internal to one team: an observation about how much of its own estimate a team completes in a Sprint. It is not comparable across teams (estimates are not a shared unit), and it does not measure value, quality or productivity. Comparing two differently sized teams on one chart produces a ranking with no meaning.
What the other measures say. Defects up 131%, NPS down 4 points, lead time flat at 14 weeks, cadence unchanged, and a growing share of capacity consumed by defect work. A 38% rise in points alongside this pattern is entirely consistent with points inflation and cutting corners: larger estimates, more items redefined as done, and less time on the work that would have prevented the defects.
So the conclusion is unsupported. The most defensible reading is that the team is producing more units of reported output while delivering no faster to customers and with deteriorating quality. "More productive" and "more output" are not the same claim.
A question worth asking. Did the Definition of Done change during these six Sprints? If yes, the two data points are not comparable and the 38% is partly an artefact of the measurement, not of the work.
2. Which measures would you put in front of the VP instead, and what current and target values would you propose?
Read the worked answer
Use the four Key Value Areas of Evidence-Based Management as the frame - Current Value, Unrealized Value, Time to Market, Ability to Innovate. One or two measures per area, chosen for this product.
| KVA | Measure | Current | Target (next quarter) |
|---|---|---|---|
| Time to Market | Median lead time, backlog accepted → production | 14 weeks | 6 weeks |
| Time to Market | Release frequency | 1 / month | 2 / month |
| Current Value | NPS | 27 | 31 (return to baseline) |
| Current Value | Adoption / usage of the last three features | unknown - instrument it | establish baseline |
| Ability to Innovate | Customer-reported defects per month | 21 | ≤ 9 |
| Ability to Innovate | % of capacity on defect and unplanned work | rising | ≤ 20% |
| Ability to Innovate | Installed version (how many customers on the latest release) | unknown | establish baseline |
Rules for using them.
- Set current values from measurement, not from memory; where unknown, the first target is a baseline.
- Targets are hypotheses to be inspected next quarter, not commitments used in performance reviews.
- Never aggregate these into one score without showing the components; the whole point is that they can move in opposite directions - as they are doing now.
Why lead time first. It is the measure here that most directly reflects value reaching customers, and 14 weeks with monthly releases is the constraint the VP can actually sponsor the removal of.
3. How should the Scrum Masters respond to the request to raise velocity another 20%?
Read the worked answer
Be direct and non-defensive. The request is reasonable in intent - the VP wants more value - and wrong in instrument. Answering "velocity is a bad metric" without offering an alternative will be heard as evasion.
The response, in order:
- Accept the underlying goal. More value delivered sooner. Say so.
- Show the trade-off already visible. Points up 38% has coincided with defects up 131% and NPS down. Squeezing another 20% in the same way predicts the same pattern at larger scale.
- Offer the alternative measures (lead time, defects, adoption, release frequency) with current and target values, and commit to reporting them next quarter in the same slot the velocity chart occupies now.
- Name one thing the VP controls. Lead time of 14 weeks is largely a function of decisions outside the teams - approval gates, release windows, environment access. Removing one of those is a bigger lever than asking teams to estimate higher.
- Protect the teams from the target. Velocity used as a target ceases to be a useful observation and becomes something to game - which is exactly what the data may already show.
Handle the retrospective question honestly. If the rise in points reflects weaker adherence to the Definition of Done, the Scrum Masters should say that to the VP and address it with the teams. Blaming the metric while ignoring the practice would be the worst of both.