Forty-Two Releases, Nine Adoptions
Ledgerline is a B2B invoicing product with six Developers, one Product Owner and a Scrum Master. Last year the team released 42 features; usage telemetry shows only nine were used by more than 5% of accounts, and the custom report builder - five Sprints of work - is used by two accounts. The Product Owner's variable pay is tied to an internal objective that counts releases shipped per quarter. Meanwhile support tickets about failed bank reconciliation have risen 30% quarter on quarter, and the two largest accounts have both raised it in renewal calls. Sprint Reviews are a slide deck of completed tickets read to eight stakeholders, four of whom never speak. The Developers say they never see a customer.
1. What is the Product Owner currently measuring, and what replacement measure of value could the team adopt within one quarter?
Read the worked answer
The team is measured on output and is therefore optimising for output. Forty-two releases with 5% adoption is not a productivity success; it is a queue of unvalidated guesses. Tying the PO's pay to releases shipped rewards shipping and punishes learning, which is the opposite of what the product needs.
A replacement measure must be (a) an outcome, not an activity, (b) influenceable by the team within a Sprint or two, and (c) checkable without asking the team's opinion. For Ledgerline: 'share of invoices reconciled without a support contact' or 'median time from signup to first successful reconciliation'. Both sit in telemetry, both connect to renewals, and both move when reconciliation is fixed.
Set one Product Goal against that measure for the quarter and hold the PO to the goal rather than the release count. One caution: choose one or two measures. A dashboard of twelve metrics restores the same behaviour in a new costume.
2. The team wants to stop work on the report builder and fix reconciliation. Sales objects. How should the Product Owner decide, and what evidence is needed?
Read the worked answer
Decide as a trade-off with numbers, and decide as the Product Owner - not by consensus in a room where the loudest voice is sales.
Evidence to gather before answering:
- Revenue at risk: the two accounts using the report builder, against the accounts raising reconciliation in renewal calls. Real ARR, not sentiment.
- Cost of delay on reconciliation: a 30% ticket rise carries support cost and churn cost, and it compounds.
- Whether the report builder needs more work at all. Two users may be served by a thin follow-up or a manual workaround.
Then bring the decision to the Sprint Review with the numbers visible, and let stakeholders argue with evidence in front of them. A sales objection is legitimate data; it is not a veto. A defensible answer is often 'reconciliation first for one quarter, and here is the date we return to reporting' - a decision with a date beats an argument without one.
3. How should the Sprint Review change so that value, not output, is what gets inspected?
Read the worked answer
Turn it back into an inspection of the product.
- Demo the working Increment against real reconciliation data, including a case that currently fails. A Review where nothing is allowed to fail is a Review that inspects nothing.
- Invite two or three actual users - a support agent, a finance manager - instead of eight silent stakeholders. Ten minutes with a real user beats an hour of slides.
- Put the value measures beside the demo: adoption of last Sprint's releases, the ticket trend. That shifts the conversation from 'what did you build' to 'what changed'.
- Keep the reporting stakeholders in the room. The goal is to change what they see, not to exclude them - and if the PO's pay is still tied to release counts, that conversation with the PO's manager is the highest-leverage one available.