The Dashboard That Says Everything Is Fine
A financial services firm reports quarterly on privacy: annual training completion 100%, DSAR timeliness 98%, privacy incidents logged 14 (all closed), and 'privacy risk rating: low'. The board's risk committee has accepted this for six quarters. Separately, a tabletop exercise found that no team could produce a complete record set for a synthetic customer within 30 days, an internal audit logged 22 medium findings of which 19 remain open after nine months, and two business units have never completed a DPIA for products launched last year. The CPO is preparing the next quarterly report.
1. Diagnose why the current metric set failed to surface these problems.
Read the worked answer
Every metric on the dashboard measures activity or closing state, not capability. Training completion, timeliness percentage, and 'incidents closed' can all be perfect while the underlying machinery is broken - and incident counts measure reporting culture as much as incident frequency, so a low number may indicate under-reporting rather than safety.
The 98% DSAR figure is the clearest example. It may be true and still miss the tabletop finding, because it measures requests that arrived through the known intake path and were closed, not whether the organization can locate all data about a person. The audit findings and the missing DPIAs never enter the dashboard at all, because the dashboard was designed around what the privacy office controls rather than what the organization depends on.
The 'privacy risk rating: low' line is the most damaging element: it is a summary conclusion with no defined method behind it, and it has trained the committee to stop asking questions.
2. What should the CPO replace it with, and what does each new measure tell the committee that the old ones could not?
Read the worked answer
Keep a small set of activity metrics for operational hygiene, but add outcome and assurance measures:
- Access completeness - result of periodic live tests: can we assemble a full record set for a sampled individual within the statutory period? This directly measures the capability the tabletop exposed.
- Remediation age - open findings by severity and days open, with an explicit threshold at which the CPO escalates. Nine-month-old medium findings are a governance failure, not a privacy failure, and the metric should make that visible.
- DPIA coverage - percentage of in-scope launches that had an assessment before go-live. Missing DPIAs are a leading indicator; a breach is the lagging one.
- Control test pass rate - sampled evidence that defined controls operate, not attestation that they exist.
Each is designed so that a bad result is uncomfortable to report. That is the point: a metric that cannot embarrass the program cannot inform the board.
3. How should the CPO present six quarters of reassuring reports alongside this new picture?
Read the worked answer
Directly, and first. If the new measures surface problems, the committee will reasonably ask why they are hearing it now, and the CPO's credibility depends on owning the answer before it is asked.
The framing that works: prior reporting measured the privacy office's throughput; the revised set measures the organization's capability. Both were accurate; the first was insufficient. Do not characterize the earlier reports as misleading - that invites a governance inquiry into the reporting itself - but do not let the shift pass unexplained either.
Then present a forward view: which measures are new, what baseline data exists (little, on day one), when the committee will see a trend rather than a point, and what the CPO is asking for. Put the two business units without DPIAs and the 19 open findings in front of the committee as named items with owners and dates, not as an aggregate rating.
A 'low' rating that survives a tabletop failure and 19 open findings is worth less than no rating at all.