Sixty deployments a week against a 2019 SLA

1. 99.95% availability per calendar month.

• Calendar-month windows reset. Two outages of over four hours that straddled a month boundary disappear into two months that each still look green. In a service that changes sixty times a week, a monthly window is far too coarse.
• A single percentage has no shape. Four short outages and one long one can produce the same number with very different customer consequences.
• Availability is undefined. Measured where, by what sampling, with what treatment of degraded performance? A service returning errors in 300 ms is "up".

2. No change-related outages.

• Zero is a binary, not a rate. It says nothing about severity, duration or how many changes were risky. One trivial defect and one four-hour outage both breach it.
• It is unfalsifiable in practice, which is why the review argues about attribution rather than about reliability. Attribution becomes a negotiation, and the measure loses authority.
• It creates the wrong incentive. Fear of breaching leads to Friday release freezes and larger, riskier batches - the opposite of what continuous delivery needs.

3. Average incident resolution time of three hours.

• A mean hides the tail. A large population of ten-minute fixes can mask a handful of incidents that took a day; those are the ones customers remember.
• It mixes severities. A severity-1 payment failure and a severity-4 cosmetic ticket are averaged together, so the number moves for reasons unrelated to service quality.
• Resolution is not restoration. Closing a ticket and restoring service are different events.

Overall. All three metrics were designed for a quarterly-release world and are being applied to a continuous one. Two of them can be green while twelve severity-1 incidents occur, which is exactly what is happening.

Design principle. Keep the SLA as the customer-facing commitment with the card schemes and regulators, and move the operating measure to SLOs that the product teams own, with an error budget that converts the target into a change-management decision.

Proposed set:

• Availability SLO: 99.95% of authorisation requests succeed, measured at the authorisation endpoint on a rolling 28-day window. Rolling windows remove the boundary effect and report the two four-hour outages immediately. Error budget: 0.05% of 28 days ≈ 20 minutes of full unavailability.
• Latency SLO: 95th percentile authorisation latency under 400 ms and 99th percentile under 900 ms. Percentiles expose the tail that the mean conceals, and latency matters to a payments customer as much as uptime.
• Change failure rate: proportion of deployments causing degraded service or requiring remediation, target below 15%, with severity recorded. Replacing the binary clause with a rate makes the measure falsifiable, removes the attribution argument if the definition is agreed in advance, and stops punishing release frequency.
• Time to restore SLO: 90th percentile restoration within 60 minutes for severity-1, reported separately from lower severities so no average mixes them.
• Incident count and severity distribution, reported alongside, so twelve severity-1 events can never be invisible.

Error budget policy. If the availability budget is more than 50% consumed in a window, non-essential change requires additional review; if exhausted, feature work pauses until reliability work restores the budget. This gives operations a real lever and gives developers a predictable rule, replacing the informal Friday freeze.

How this period would have reported. Availability would have shown the two long outages as an immediate budget burn rather than two green months. Change failure rate would read 50% of severity-1s (6 of 12), an unambiguous red. The restoration percentile would have surfaced the long tail despite the 2h40m mean.

Remove the ambiguity before the meeting, not in it.

The argument about which changes count as change-related exists because the definition was never written down. Convene a short, separate session with the product owners, operations lead and SLM to produce a measurement specification: what counts as a change, what counts as a failure, how severity is assigned, where each metric is sampled, what the reporting window is and who arbitrates a disputed classification. Publish it. Any residual dispute is then a change request against the specification, not a recurring agenda item.

Change the meeting's inputs. Monthly packs reviewed after the fact cannot support decisions in a service changing sixty times a week. Publish SLO dashboards continuously and make the review's input the error budget position and the trend, not a static slide of green ticks. Everyone arrives having already seen the numbers, so the meeting is spent on causes and actions.

Change the attendees and the agenda. The service owner should not present to the product teams; the product owners and operations lead should co-own the review. A workable agenda: error budget status and burn rate over the last four weeks; the severity-1 incidents and their contributing causes; the change failure rate and what the failures had in common; open improvement actions with owners and dates; and one forward-looking item - the riskiest change planned in the next period.

Close the loop. Every action gets an owner, a date and a measurable definition of done, recorded in the improvement register. The next review opens with the previous actions, not with the dashboard. If an action is not closing, the review should ask why rather than re-listing it.

Watch for gaming. Once change failure rate is measured, it will be argued about. Spot-check classifications, keep the definition stable for at least two quarters, and resist adding new metrics until the current set is trusted and being acted on.