DevOps Measurement, Metrics and Resilience
Domain 4 - DevOps Measurement, Metrics and Resilience
This domain tests metric selection and resilience engineering. Scenario questions ask which measure reveals system improvement, how to use error budgets, and how to respond to incidents without creating blame.
Measurement system
A good measurement system is transparent, owned by teams, and used for learning. It should show whether a change improved flow or resilience, not rank people. Review metrics alongside qualitative evidence from incidents and value-stream maps.
Metrics that drive behavior
- Use flow and outcome metrics: lead time for changes, deployment frequency, change failure rate, and time to restore service.
- Avoid vanity metrics and individual utilization; they reward local efficiency and queue growth.
- Pair speed with stability. A team that deploys more often but fails more is not improving.
- Define SLIs from the user's perspective, set SLOs, and manage an error budget. The budget makes reliability and release velocity a shared trade-off.
- Measure detection and recovery, not just failure counts.
Resilience practices
- Design for failure: redundancy, graceful degradation, circuit breakers, timeouts, and rollback.
- Use canary releases and dark launches with observability to limit blast radius.
- Run blameless incident reviews that identify systemic causes and track actions.
- Practice failure injection and game days to test assumptions.
Worked example
A service has a 99.9% availability SLO. The error budget is exhausted after repeated failed releases. The team wants to speed up. Correct response: pause risky feature releases, prioritize reliability work, use canary deployment with automatic rollback, and measure time to restore. Reasoning: the SLO converts reliability into an explicit budget; spending it on more failed releases is not speed. The metric to watch is change failure rate and restore time, not deployment count alone.
Traps that cost marks
- Targeting 100% availability, which removes the error budget and encourages risk aversion.
- Using MTTR as a blame metric rather than a system measure.
- Reporting deployment frequency without change failure rate.
- Confusing monitoring dashboards with observability.
- Treating postmortem actions as optional.