Evaluation, Monitoring, and Improvement
Two different jobs
Monitoring is continuous and process-focused: are we doing what we said, and are the early signals moving? Evaluation is periodic and judgement-focused: did it work, for whom, and at what cost? Formative evaluation informs mid-course correction; summative evaluation judges the finished result.
Concepts you must hold
- Outputs, outcomes, impacts: outputs are what was delivered (sessions run, records processed); outcomes are the changes those outputs produce; impacts are longer-term consequences. Most weak evaluations measure outputs and call them results.
- Indicators: specific, measurable, baselined, and targeted. An indicator without a baseline cannot support a claim of improvement.
- Data quality: consistent definitions, complete collection, and stable measurement over time. Changing the definition mid-stream invalidates comparison.
- Improvement cycles: plan–do–study–act and similar continuous-improvement models, with small tests before full-scale change.
- Variance analysis: when results diverge from plan, diagnose the cause rather than simply restating the gap.
- Corrective versus preventive action: fixing a detected problem versus removing the conditions that would create it.
- Lessons learned and sustainment: capturing what worked, embedding it in routine practice, and checking the gain holds once attention moves elsewhere.
Traps
Reporting activity as achievement. Claiming improvement without a baseline. Re-baselining or redefining the measure to make results look better. Assuming a completed improvement project equals an improved system. Using vanity metrics that always rise.
Rule of thumb
No baseline, no claim. Before accepting any statement of improvement, ask what the starting value was, how it was measured, and whether the definition stayed constant.
Worked example
A report celebrates 100% training completion. That is an output. If error rates fell from 8% to 3% over the same period, with a stable definition and a comparable baseline, that is an outcome worth reporting - and even then, alternative explanations should be considered before attribution is asserted.