The 59% Improvement
You run safety analytics for a 900-vehicle logistics company. In January the company replaced a 12-field paper near-miss form with a three-tap mobile form that drivers can complete at the depot gate. Reported near-misses fell from a 46-per-month average to 19. The CEO has already circulated a slide celebrating a 59% safety improvement, and two of your largest customers have asked for the figure for their supplier scorecards. Three facts sit next to the headline: driver headcount is down 8% year on year; the new form's 'other' category now absorbs 31% of submissions, up from 6%; and dispatch supervisors, who previously filed reports on drivers' behalf, have no submission path in the new system. The fleet's telematics data shows hard-braking events are flat.
1. Is the 59% figure defensible? What do you say to the CEO before the customer meetings?
Read the worked answer
The arithmetic is right; the inference is not. 46 to 19 is a 58.7% fall in reported near-misses. Exposure explains only a sliver: adjusting for the 8% headcount reduction still leaves roughly a 55% per-driver fall. So the drop is real as a count - but the count is of reports, not of risk.
The two mechanism signals point to suppressed capture, not reduced hazard. When 31% of submissions land in 'other' (up from 6%), the form is failing to classify what people see. When supervisors lose their submission channel, an entire reporting route disappears - and supervisors were previously filing on drivers' behalf, which is exactly the population whose reports would vanish. Meanwhile hard-braking is flat: an independent measure of risk that did not move while your headline measure fell by more than half.
What you say: bring one sentence the CEO can still use - 'reported near-misses per driver fell 59% after the form change' - and one sentence of caution that the fall is partly a reporting artifact, with the three facts and a plan to resolve it in ten days. Do not refuse to give a number; a refusal leaves a vacuum the CEO will fill with the worse version.
2. What analysis separates a reporting-behaviour change from genuine risk reduction, and what is the minimum you can credibly deliver in ten days?
Read the worked answer
Three lines of evidence, cheapest first.
- Capture reconciliation. Restore the supervisor submission path, then run one month of dual reporting on the old and new forms for a sample of depots and compute a capture ratio. Even a rough ratio converts your headline into a corrected estimate with a stated range.
- Independent risk measures. Hard-braking is already flat; add damage claims per 1,000 km and injury claims per 100 drivers. If true risk fell by half, some of these should have moved.
- Severity mix. A genuine risk reduction usually shows up across severities. A fall concentrated entirely in low-severity reports, with injuries flat, is the classic signature of a reporting artifact.
Ten-day minimum: publish a written metric definition change; restore the supervisor channel retrospectively by one week so the gap is measurable; deliver one reconciled month with a capture ratio and an explicit range.
What you must not promise: a causal estimate of the programme's safety effect. Ten days gets you a corrected descriptive number and a list of what you still cannot say. Say that plainly now rather than after the customer meetings.
3. A customer scorecard asks for 'recordable incident rate.' What do you supply?
Read the worked answer
Supply the recordable rate from the incidents and claims system - not the near-miss count relabelled. Near-misses and recordable incidents are different constructs with different thresholds and different legal definitions. Passing one off as the other is a misrepresentation to a third party, and it is the kind of misrepresentation that surfaces in an audit two years later.
Send three things: the recordable rate with its definition, its period, and its denominator; the near-miss figure separately and explicitly labelled 'reported near-misses, self-reported via mobile form'; and a one-line note that the two moved in different directions after a January reporting-process change, which is itself material information for a supplier assessment.
If the customer insists the near-miss number go into their template field: supply it with a footnote in the field itself defining what it is. Never let the label do work the data cannot support.
Why this is the harder call rather than the easier one: the customer wants a number and the near-miss number is available and looks better. The temptation is to treat 'recordable' as a formatting question. It is a definitional one, and the person asking is not the person who will be accountable when someone checks.