A 40-indicator scorecard after a supervisory inspection

The scorecard responds to a finding about weak oversight with volume rather than decision quality. Forty indicators cannot be discussed in a board meeting; most are operational counts without thresholds; none maps to the four findings; and leading and lagging measures are mixed without distinction. Helpdesk ticket volume measures demand, not resilience.

Design backwards from the decisions the board must take: accept a risk, fund remediation, escalate a provider, delay a change. Then select indicators that trigger those decisions.

Core set, each with a threshold, an owner and an escalation route:

• Risk appetite utilisation against stated tolerance for availability, integrity and third-party concentration.
• Tested coverage of critical functions versus plan, plus critical functions untested and their age.
• Improvement actions overdue, average age, and number past the escalation threshold.
• Major incidents, repeat incidents, classification accuracy and notification timeliness.
• Critical contracts without required clauses, concentration exposure, providers with unresolved findings.
• Open testing findings by severity and age, with retest closure rate.
• Significant changes made without a corresponding review of the testing plan, recovery arrangements and provider dependencies - this one addresses the inspection finding directly.

Leading versus lagging. Overdue actions, ageing findings and unreviewed changes predict incidents; downtime and incident counts report them after the fact. Boards act on the former, so they should dominate the pack.

Agree the set with the board, document why each indicator exists and what decision it triggers, and review the set annually. A scorecard unchanged for two years is itself evidence against continual improvement.

A closed loop with records, not a statement of intent. The loop: detect (metrics, incidents, testing, audit, supervisory findings) → analyse (root cause, trend, updated risk assessment) → decide (management body or delegated committee, minuted) → act (owner, budget, date) → verify (retest or audit) → update the framework.

Applying it to the four findings:

• Testing plan not updated after the platform change. The most serious finding, because it proves the change-to-resilience link is broken. Introduce a mandatory trigger: defined categories of change require a documented review of testing scope, recovery arrangements and provider dependencies before go-live. Evidence is the completed reviews for the next N changes, not a new policy paragraph.
• Two overdue actions without escalation. Define escalation by age and risk, hold improvement actions in the same register as incidents and audit findings, and report overdue items to the board with names attached.
• Risk appetite not reviewed since 2022. Annual review is the minimum, plus review on significant change. Ratify at the management body with minutes.
• Availability-only dashboard. Replaced by the redesigned scorecard above.

Verification capacity. The inspection found internal audit cannot challenge management. Co-source or train, and mandate audit to test the improvement loop itself, not only individual controls. Without challenge capacity the loop decays within a year.

Cadence. Schedule the annual framework review so the cycle has a fixed owner and date, and align the eleven-month self-assessment to produce changed artefacts with dates - revised testing plan, revised appetite, closed actions with evidence - rather than a rewritten narrative.

Rank by risk, not by ease of fixing.

1. Testing plan gap (0-30 days). The only finding with a live, unmitigated exposure. A core platform changed and assurance did not follow, so nobody knows whether recovery and testing assumptions still hold. Re-scope testing for the changed components immediately, run a scenario test, confirm recovery arrangements, and check whether data flows to providers changed.

2. Overdue improvement actions (0-60 days). A governance failure that will recur and will discredit everything else. Re-plan with owners and budget authority, apply the new escalation rule, and root-cause why they stalled - usually no funding authority, which is a management body issue rather than a project issue.

3. Risk appetite and dashboard (30-90 days). These are the levers that prevent recurrence, but they take longer. Sequence matters: the scorecard is meaningless before appetite is set, so ratification precedes dashboard redesign. Doing them in the wrong order produces metrics with nothing to measure against.

4. Audit capability (parallel, 3-6 months). Without independent challenge, items 1 to 3 will decay. Co-source or train now so the next cycle is tested by someone credible.

Presentation to the regulator. Distinguish quick fixes from structural change, give dates, and do not present all four as equally urgent. A response treating them as equivalent reads as an absence of risk judgement - precisely what the inspection criticised. State explicitly which finding carries live exposure, and what compensating monitoring applies until the structural fix is verified.