Twenty Deploys a Day, Same Three Incidents

Deployment frequency is a flow output, not a learning outcome. The same three incident types recur because the organization has no effective feedback loop from incidents to changed behavior. Postmortem actions are 64% overdue and are mostly vague: 'add to runbook' or 'be more careful' do not change system conditions. The Confluence space is a write-only archive; 600 pages and 12 views show that storing lessons is not the same as learning them. Knowledge is concentrated in two senior engineers, so the organization depends on heroes rather than building capability. Training was cut and game days were canceled precisely when the repeat rate shows learning is needed. The pipeline being fast makes it easier to deploy, but without feedback it also makes it easier to repeat the same failure. Continual learning requires experiments, practice, and follow-through, not more documentation. The 31% repeat rate is the key signal that the program has optimized flow at the expense of feedback.

Use a small set of connected practices. First, require postmortems for the three incident types with actions that change a system, not a person: automate certificate renewal, add connection-pool saturation alerts and load tests, and add feature-flag validation in the pipeline. Each action gets one owner and a due date, and is reviewed at 30 days. Second, run monthly game days for one failure mode at a time, starting with TLS expiry, in a controlled environment, and rotate participants so the two senior engineers coach rather than solve. Third, introduce chaos experiments only after the game day is routine, with a hypothesis, blast-radius limit, and abort condition. Fourth, create a short learning review every two weeks where teams share one change they made and one assumption that was wrong. Measure repeat-incident rate, on-time action closure, time for a new engineer to lead an incident, and the number of game-day findings that become automated checks. Do not measure page views. The mechanism must close the loop from incident to system change to verification.

Frame the case in the CFO's language: repeat incidents are 31% of all incidents, mean time to restore and on-call concentration create cost and risk, and 14 weeks to independent on-call is a capacity constraint. Estimate the cost of the three recurring incident types, including engineering time, customer credits, and delayed releases. Then propose a bounded pilot, not a broad chaos program. Start with one failure mode, TLS expiry, in a staging environment or a low-risk production service with a defined hypothesis, blast radius, abort condition, and rollback. Measure time to detect, time to restore, and whether the fix prevents recurrence. Ask for a small budget for game days and training, tied to reducing repeat incidents by a target percentage and cutting new-engineer time to independent on-call. Reframe chaos engineering as controlled testing of known weaknesses, not 'breaking production on purpose.' If the pilot does not reduce repeat incidents or restore time, stop and report that. The CFO needs an exit criterion, not a promise of a learning culture.