Solution Evaluation
Purpose
Solution evaluation judges whether a delivered or partial solution delivers the intended value. It covers measure solution performance, analyze performance measures, assess solution limitations, assess enterprise limitations, and recommend actions to increase value.
Concepts to understand
Performance is measured against the business goals and objectives defined earlier, using defined metrics and baselines. Analysis asks why results differ from expectations. Limitations come in two kinds: solution limitations (defects, gaps, usability, missing capability) and enterprise limitations (culture, policy, skills, process, incentives). Recommendations may include fixing the solution, changing the organization, or retiring the solution. Evaluation applies to solutions in use, not only to finished projects - it can start as soon as a component is deployed.
Traps
The key trap is attributing every shortfall to the solution when the cause is organizational. Another is measuring activity (users trained) instead of outcome (error rate reduced). A third is evaluating only at project close, missing the chance to correct course earlier. Also watch for recommendations that ignore the cost and risk of the fix itself.
Rule of thumb
Measure against the original objectives; when results fall short, ask first whether the limitation is in the solution or in the enterprise.
Worked example
A new self-service portal meets its technical targets but adoption is low. Solution limitation? Users report the flow is clear. Enterprise limitation? Front-line staff still direct customers to the call center because their targets reward call handling. Recommendation: adjust incentives and messaging, not the portal.