The green dashboard that hid the problem

The measures are mostly activity or output indicators, not effectiveness indicators. Training completion shows that people clicked through a module; it does not show that they recognise phishing, report incidents or follow secure practices. Zero reported incidents is ambiguous: it may indicate a low incident rate, but combined with staff saying they did not report a suspicious email, it more plausibly indicates under-reporting. A lack of reports is not evidence of security; it may be evidence of a weak reporting culture.

Patch compliance at 98% is misleading without a defined population. The branch server was outside the scanning tool, so it was excluded from both the numerator and the denominator. The metric measured the estate that was easy to measure, not the estate that mattered. 'All 93 controls implemented' is a statement about the SoA, not about operating effectiveness; a control can be documented and still fail in practice.

The dashboard also lacks segmentation and trend information: which business units, which asset groups, which risk categories. A single aggregate can hide a failing branch or a systemic privilege problem. The CEO's conclusion was reasonable given the data, which is precisely the problem: the measurement design gave false assurance.

Management review should have challenged the definition, source, coverage and limitations of each metric. For training, it should ask whether completion is linked to behaviour and whether phishing simulations or reporting rates support the claim. For incidents, it should ask whether reporting channels are trusted and whether near-misses are captured. For patching, it should ask which assets are excluded and how coverage is assured. For the SoA, it should ask for evidence of operating effectiveness, not just implementation status.

Now it should direct a corrective action programme: bring the branch into the scanning and patch-management scope; remove unnecessary local administrator rights and introduce privileged access management; eliminate the shared finance password and replace it with individual accounts and a secrets-management approach; and investigate why staff do not report suspicious email. It should review the awareness programme's design and measure behaviour change, not poster counts.

The review should also examine whether the CISO had the resources and access needed to measure the whole estate, and whether bad news is rewarded when it is reported. Decisions, owners, dates and required resources should be recorded, with follow-up at the next review.

Clause 9.1 requires the organisation to determine what needs to be monitored and measured, the methods, when monitoring and measurement will be performed, who will analyse and evaluate the results, and when. Bright & Co should rebuild its measurement framework around those questions.

First, define a balanced set of indicators: coverage (percentage of assets in scope of scanning), effectiveness (percentage of tested controls that operate as intended), behaviour (phishing report rate and repeat-click rate), and outcomes (confirmed incidents, time to detect, time to respond). Second, validate the data sources and document exclusions; a patch metric must state the asset population and how completeness is verified. Third, assign analysis responsibility and a review cadence, with thresholds that trigger escalation. Fourth, compare results with risk criteria and previous periods, not just with targets. Fifth, retain evidence of monitoring and use the results in management review and continual improvement.

A useful test is whether the metric could reveal bad news. If every indicator is designed to go green when activity is completed, the framework measures busyness rather than risk. Adding posters would repeat the original mistake: it addresses awareness output, not the reporting culture or the control gaps the penetration test exposed.