SRE Practitioner key terms

Manual, repetitive operational work that scales linearly with service growth, produces no lasting value, and is a prime target for automation so engineers can spend time on engineering work.

A structured review after an incident that focuses on systemic causes and contributing factors rather than punishing individuals, on the premise that people act reasonably given the information they had.

Desensitization that occurs when responders receive too many low-value or duplicate alerts, causing real signals to be missed and slowing detection and response during genuine incidents.

Forecasting future demand for a service and provisioning enough resources ahead of time so that performance objectives hold under expected growth and peak loads without wasteful overprovisioning.

The amount of unreliability a service may experience in a period, usually one minus the SLO, which teams spend on releases and risk-taking and stop spending when exhausted.

A quantitative measure of some aspect of service behavior from the user's perspective, such as request latency, error rate, throughput, or data freshness, expressed as good events over valid events.

A target range for a service level indicator over a defined window, for example 99.9 percent of requests faster than 300 ms over 28 days, used to judge whether reliability is acceptable.

A contract with customers or internal stakeholders stating promised service levels and the consequences of missing them, usually set looser than the internal objective that protects it.

How fast a service consumes its error budget relative to the rate that would exhaust it exactly at the end of the window; multi-window thresholds drive alerting on fast and slow burns.

A pre-agreed written rule describing what happens when a service exhausts its error budget, such as halting feature releases and prioritizing reliability work, signed off by product and engineering leadership.

Four core metrics for monitoring a user-facing system: latency, traffic, errors, and saturation. They give a quick, broadly applicable view of health before deeper instrumentation is consulted.

A technique that follows a single request across service boundaries by propagating a shared identifier, producing a timeline of each component's contribution to end-to-end latency and failures.

A single named, timed operation within a trace, carrying start and end times, attributes, and parent-child relationships that show where work happened inside a request.

Passing trace and span identifiers through every call, message, and queue between services so that independently instrumented components can be stitched into one coherent trace.

The number of distinct label or attribute combinations a metric series can take; high cardinality multiplies storage and query cost and is a common cause of monitoring system overload.

A practice where engineers define and instrument the signals a new feature must emit before or alongside writing it, then use production telemetry to validate behavior and guide iteration.

A security model that assumes no implicit trust based on network location and instead authenticates and authorizes every request, using least privilege and continuous verification of identity and device posture.

Granting each user, service, or process only the permissions needed for its current task, for the shortest time possible, to limit what an attacker or a bug can reach.

The scope of harm a failure or compromise can cause, measured in affected users, services, or data; designing to contain it is central to both resilience and safe change.

Layering independent security controls so that a single failure or bypass does not lead to compromise, relying on overlapping prevention and detection rather than one strong perimeter.

Designing systems to keep serving a reduced but useful function when dependencies fail, for example returning cached or partial results instead of errors.

The single person accountable for directing a major incident: setting priorities, assigning roles, coordinating communication, and deciding when to escalate, while others do the hands-on debugging.

The average duration from incident start or detection to restoration of service; a key outcome measure of response capability, tooling, and system design.

An event with significant customer or business impact that requires coordinated, cross-team response, formal communication, and usually a declared severity with an assigned commander.

A step-by-step documented procedure for handling a known alert, failure, or routine operation, kept current so responders act quickly and consistently under pressure.

The disciplined practice of running controlled experiments that inject realistic failures into a system to discover weaknesses before they cause real outages in front of customers.

A measurable statement about normal system behavior, such as a given success rate, that an experiment expects to remain true while a fault is injected; deviation reveals weakness.

Deliberately introducing conditions such as latency, errors, resource exhaustion, or instance loss into a system, in test or production, to observe how it responds.

A planned exercise where a team simulates an incident or injects failures to rehearse response, validate runbooks and tooling, and surface gaps in people, process, and technology.

A curated set of self-service tools, templates, and infrastructure abstractions that lets product teams build, deploy, and operate services consistently without depending on a central operations team.

An opinionated, well-supported default way to build and ship a service on the platform, covering templates, delivery pipelines, observability, and security, so most teams need not invent their own.

Deploying a change to a small subset of traffic first and comparing its health signals against the baseline before widening rollout, containing the blast radius of a bad release.

Applying machine learning and statistical analysis to operational telemetry to correlate events, detect anomalies, and suggest or automate diagnosis and remediation during incidents.

Automatically flagging data points that deviate from learned or expected behavior, used to catch incidents that static thresholds miss and to reduce noisy rule maintenance.

Applying agile and DevOps practices, including versioning, automated testing, continuous delivery, and monitoring, to data pipelines so analytics and machine learning inputs stay timely and trustworthy.

A schedule that assigns engineers primary and secondary responsibility for responding to production alerts during defined hours, balanced for load and paired with clear escalation paths.

A team climate where people can raise concerns, admit mistakes, and challenge decisions without fear of punishment, which underpins blameless learning and honest incident reporting.

A structured checkpoint before a service takes real traffic, covering reliability, observability, security, capacity, and on-call ownership, against agreed criteria that must be met.