Six Months In, the Sponsor Wants to Declare Success
Six months after go-live of a self-service procurement portal, the sponsor wants to declare success. The business case promised purchase-order cycle time falling from 9.2 to 4.0 days and adoption by 70% of 480 buyers within six months. Actuals: cycle time 6.8 days, adoption 41%, satisfaction 3.1 out of 5, and 220 support tickets per month, most reporting that approval status cannot be found. Two regions received training six weeks late because of a scheduling conflict, and in one region single sign-on was deferred to a later release. The benefit model assumed full adoption across all regions and used cycle time measured from requisition release to PO issue. You have been asked to write the benefits evaluation.
1. Can you conclude the solution underperformed? Set out how you would evaluate before drawing a conclusion.
Read the worked answer
No - the aggregate numbers cannot support that conclusion, and neither can they support the sponsor's. Two distinct questions are being conflated:
- Solution performance: does the portal deliver 4.0-day cycle time where it is actually used?
- Benefit realization: are enough buyers using it, correctly, for the benefit to accrue?
Segment before concluding. Split all four metrics by region, and within region by training cohort (on-time vs. six weeks late) and by SSO status (integrated vs. deferred). The hypothesis to test is that active users show near-target cycle times and the aggregate is dragged down by low-adoption segments. If so, this is a deployment and adoption shortfall, not a solution defect - and the corrective actions are entirely different.
Validate the measures themselves. Confirm cycle time is being calculated on the same definition used in the baseline (requisition release to PO issue). If the baseline was measured from a different point, or measured on a pilot group of motivated users, the 9.2-day comparison is not like-for-like and must be restated before any variance is reported.
Treat the 220 tickets as data, not noise. "Cannot find approval status" is a recurring usability finding with a traceable cause; it plausibly explains both the satisfaction score and some adoption resistance.
2. What do you recommend, and what evidence would change your recommendation?
Read the worked answer
Recommend a two-track response, with a defined re-measurement point.
Track 1 - remove adoption blockers (short term, low cost):
- Approval-status visibility: this is a design/requirements gap surfaced by real users. Route it through change control as a small enhancement; it is likely the single highest-leverage fix given the ticket volume.
- Complete the delayed training in the two regions and confirm competency, not just attendance.
- Deliver the deferred SSO integration in the next release and re-measure that region afterward.
Track 2 - re-measure in 90 days against the same definitions, with thresholds agreed in advance (e.g., adoption ≥ 60% and cycle time ≤ 5.0 days among trained users as interim targets toward 70%/4.0).
The evidence that would change the recommendation is the segmented cycle time: if buyers who are trained and using the portal consistently show ~4.1 days, the benefit is achievable and the problem is adoption. If active, trained users are still at 6.5 days, the solution itself underperforms its design intent and requires a product change, not a training campaign. Settle that question before committing further spend.
Also state plainly whether the 70% adoption assumption was ever validated with the buyer population, since benefit models frequently assume adoption that no one has tested.
3. How do you report to a sponsor who wants to declare success, without either capitulating or becoming an obstacle?
Read the worked answer
Report against the framework agreed at baseline, and say so up front. Use the original measures and definitions; do not quietly redefine cycle time or adoption to produce a better number, and do not accept a redefinition proposed now to do the same. If the baseline itself was flawed, present the restated baseline separately and label it as a change, with rationale.
Structure the report in three parts:
- What was delivered - the capability is live, N users onboarded, these functions working. This is true and should be stated plainly; it is not the same as benefits realized.
- What the benefits actually are, disaggregated - actual vs. target on cycle time, adoption, satisfaction, with the regional splits and the training/SSO factors. Variance explained, not just variance reported.
- What happens next - the corrective actions, the 90-day re-measurement, and the residual risk if the sponsor chooses to close the evaluation now.
Handle the declaration directly but without confrontation: offer the sponsor two defensible positions - "capability delivered, benefits at risk, corrective plan in place" now, or "benefits evaluated" in 90 days. A declaration of success made against a 41% adoption figure will not survive the next audit or the next budget cycle, and that argument usually lands better than an appeal to methodology.
Record the decision either way, including the sponsor's acceptance of residual risk, so the evaluation close-out reflects what was known at the time.