The Insurer That Bought a Toolchain Instead of a Culture
A regional insurer has run a 14-day release train for six years. Development sits on floor 3, operations on floor 5, and work crosses between them only through a ticket queue. Every production change needs sign-off from a weekly Change Advisory Board that meets on Tuesdays; a change approved on Wednesday waits six days. Last quarter the change failure rate was 31% and mean time to restore was 9 hours. After a payments outage traced to a misconfigured connection pool, the post-incident report concluded "operator error - individual retraining assigned", and the engineer involved was moved off the team at his manager's request. The new CTO has announced a DevOps transformation and asked for a toolchain purchase: GitLab, Terraform, Kubernetes and an observability suite. The budget is approved. Attrition in operations is 22% annualised, and 60% of respondents in the last pulse survey said they would not report a mistake. The CTO wants a 90-day plan.
1. The CTO frames the transformation as a toolchain purchase. Evaluate that framing, and identify the two signals in the situation that most threaten the programme.
Read the worked answer
Reasoning. Tools do not create collaboration; they automate whatever process exists. If the current process is a six-day approval queue between two floors, GitLab and Kubernetes will produce a faster, more reliable handoff, not a shared ownership model. The organisation will get automated gates with the same trust deficit baked in - and will conclude DevOps "didn't work" when lead time barely moves.
The two threatening signals.
- The post-incident outcome. A report that stops at "operator error" is a finding about the system's design, not the person. The misconfigured connection pool was reachable, untested and undetected - that is a guardrail and observability gap. Moving the engineer off the team converts one incident into an organisation-wide lesson: concealment is safer than disclosure. With 60% of staff already saying they would not report a mistake, this single action outweighs the entire tool budget.
- The CAB as the only trust mechanism. A weekly board exists because the organisation does not believe changes are safe by default. Buying CI/CD without replacing that control produces a pipeline that terminates in a Tuesday meeting.
Secondary but material: 22% operations attrition destroys the tacit knowledge that makes a 9-hour restore possible at all.
Conclusion. Fund the tools, but sequence culture first: the 90-day plan must change how incidents are reviewed and how change risk is classified before or alongside the first pipeline. A useful test - if the CTO cannot say what behaviour will be different on day 90, the purchase is a distraction.
2. Propose the first 90 days of interventions, in order, with the rationale for the sequence and one measurable indicator per intervention.
Read the worked answer
Sequencing principle. Start with the interventions that unlock disclosure, then the ones that remove structural blockers, then the ones that build new habits. Doing them in the reverse order means the new habits are built on a foundation that punishes honesty.
Days 1–30 - Make it safe to speak.
- Rewrite the post-incident template: contributing factors, detection gaps, why the guardrail did not exist. Explicitly prohibit naming individuals as causes. Re-run the payments outage review under the new template and publish it.
- Indicator: percentage of reviews containing a contributing-factor analysis rather than a named person (target 100%).
- Publish a written "no blame for error, accountability for concealment" statement signed by the CTO.
- Indicator: pulse-survey question "I would report a mistake I made" - baseline 40%, target 65% at day 90.
Days 31–60 - Remove the structural blocker.
- Introduce risk-tiered change classification: low-risk changes (config, flag flips, tested code behind a flag) are approved by peer review at merge time; only high-risk changes go to a board. This is the control that makes a pipeline meaningful.
- Indicator: median lead time from merge to production for low-risk changes.
- Embed one operations engineer into each product squad, with on-call shared both ways. Indicator: share of changes reviewed by someone who also carries the pager.
Days 61–90 - Build the habit.
- Run a value-stream mapping on one service end to end, and start one team on a real pipeline as a reference implementation, not a pilot that gets abandoned.
- Indicator: deployment frequency and change failure rate for that one team.
Why this order. Disclosure without a removal of the CAB creates frustration; removing the CAB without disclosure creates unmanaged risk; a pipeline without either creates expensive theatre.
3. The CTO wants a single "DevOps maturity score" out of 100 to report to the board each quarter. Critique this request and propose what to report instead.
Read the worked answer
Critique. A single composite score is attractive to a board and corrosive to the programme, for three reasons.
- It is gameable. Any weighted index invites optimisation of the inputs - running more pipelines, writing more tests, closing tickets - rather than outcomes. The moment a score becomes a target, it stops measuring anything.
- It hides the trade-off. The four DORA measures behave as two pairs: deployment frequency and lead time measure speed; change failure rate and time to restore measure stability. A team can raise a composite score by deploying more often while change failure rate climbs. The composite cannot show that, and the trade-off is precisely the interesting conversation.
- It implies a destination. Maturity models rank organisations against someone else's baseline. The useful question is not "how mature are we" but "is our delivery capability improving against our own trend, and where is the constraint".
What to report instead.
- The four DORA measures as a trend over at least four quarters, per value stream, with the teams' own baselines - never as a cross-team league table, which produces sandbagging.
- One or two outcome measures the board actually cares about (e.g. availability against an agreed SLO, or time to deliver a customer-visible change).
- The culture indicators, because they predict the rest: incident-reporting willingness, on-call attrition, share of changes reviewed by a peer who also carries the pager.
If a single number is politically unavoidable, report the trend and direction of a deliberately narrow measure (e.g. lead time for low-risk changes) and publish the component data alongside it. Refuse to compare the insurer's score to an industry benchmark derived from a different context.