Prometheus Certified Associate common mistakes
Prometheus Certified Associate · Understand the traps and how to avoid them.
Computing a rate over a window shorter than the scrape interval, e.g. rate(http_requests_total[30s]) with a 60s scrape interval.
Candidates treat range selectors as arbitrary lookback windows and assume more samples always means a smoother result. With fewer than two samples in the range, rate() and increase() return no data points, so panels and alerts silently go empty. This surfaces most often when someone copies a short window into a recording rule or alert expression.
What to do instead
Keep the range selector at least 4x the scrape interval (the docs' recommendation), e.g. rate(http_requests_total[5m]) with a 1m scrape. When debugging empty results, check the metric's actual sample spacing, not the dashboard range. Use a recording rule at a stable interval if you need short-window rates.
Using a raw counter in a graph or alert, or applying rate() to a gauge.
The metric type is not visible in the UI, so candidates pick metrics by name and forget that counters only grow (and reset on restart), while gauges go up and down. A raw counter graph looks like a staircase and a counter-based alert fires forever once the threshold is crossed; rate() on a gauge produces meaningless noise.
What to do instead
Before writing an expression, confirm the type via the metric's HELP/TYPE metadata, the exporter docs, or by querying the series over time. Use rate/increase/irate only on counters; use avg_over_time, max_over_time, deriv, or delta-style reasoning on gauges. Remember rate() already handles counter resets.
Aggregating with sum() before dividing, or forgetting to aggregate at all, when computing an error ratio, and then summing ratios across instances.
Candidates reason about the ratio per request stream, so they write sum(errors)/sum(total) incorrectly as an average of per-instance ratios, or add up ratios from several pods which can exceed 1. The classic mistake is averaging percentages instead of dividing summed numerators by summed denominators.
What to do instead
Aggregate the numerator and denominator separately, then divide: sum(rate(http_requests_total{code=~"5.."}[5m])) / sum(rate(http_requests_total[5m])). Keep labels you want to group by on both sides (e.g. by (job, instance)) so the division matches. Guard against division by zero with a clamp or a > 0 filter on the denominator.
Assuming rate() returns a per-second value aligned to the query step, and misreading the value of increase() as an integer count.
Candidates expect rate to be a simple difference over the window. In fact rate() extrapolates to the window boundaries and increase() can return fractional values (e.g. 2.5 requests) because of interpolation and counter resets. Alerts written as increase(x[5m]) > 10 therefore behave differently than expected near window edges.
What to do instead
Treat rate() as an extrapolated per-second average and increase() as an extrapolated total, both approximate. For alerts, prefer rate() thresholds with a for: duration, or round/increase carefully and document the approximation. If you need exact per-interval counts, scrape at the same interval and use increase() with a range that is a multiple of the scrape interval.
Matching labels incorrectly: using = with a regex, forgetting that =~ is fully anchored, or joining two metrics with a bare on() that drops the labels you need.
PromQL's regex matchers are anchored by default (env=~"prod" does not match "prod-eu"), and vector matching defaults to all shared labels. Candidates coming from SQL expect substring matching and loose joins, so filters silently match nothing or many-to-many errors appear.
What to do instead
Use env=~"prod.*" for prefix matching and env!="" to test presence. When combining metrics, write explicit on(job, instance) or ignoring(...) and check for duplicate series with count by (...) to catch many-to-one errors. Use label_replace or label_join deliberately, and test expressions in the Prometheus /graph page before shipping.
Writing alert rules without a for: clause and with thresholds that fire on a single scrape, then being surprised by flapping or missing a real incident because the alert never resolves.
Candidates focus on the expression and treat for: as optional decoration. A one-sample spike or a counter reset produces a page, and alerts without a resolve path stay firing. Similarly, an alert using a raw counter never returns to false, so it never resolves.
What to do instead
Add for: 5m (or longer for noisy signals) to require sustained breach, and base expressions on rate() so they fall back below threshold. Give every alert a summary and a runbook annotation, and verify the resolve path by checking the alert's state (pending/firing) in the Prometheus /alerts page after a test.
Instrumenting a counter for a value that can decrease (queue depth, in-flight requests, temperature) and then wondering why rate() shows negative or reset-like spikes.
The four metric types exist for a reason, but candidates default to Counter because it is the first type in most client libraries. Client libraries may reject negative increments on counters, or the value resets cause rate() to interpret the drop as a counter reset and inflate the rate.
What to do instead
Choose the type from the semantics: Counter for monotonic totals (requests, errors, bytes), Gauge for current values, Histogram for latency/size distributions, Summary only when you cannot pick buckets. Name counters with a _total suffix, and if you need a rate of a gauge, use deriv() or delta-style logic on a gauge, not rate().
Creating histograms with default or poorly chosen buckets, then computing quantiles with histogram_quantile(0.99, ...) and trusting the result.
Candidates assume histogram_quantile returns an exact quantile. It interpolates within a bucket, so a 99th percentile above the highest bucket boundary is reported as +Inf or as the top boundary, and default buckets (0.005 to 10) are wrong for a service whose latency lives in milliseconds or tens of seconds.
What to do instead
Define buckets around your SLO boundaries (e.g. 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5) and always aggregate with sum by (le) before histogram_quantile. Check the +Inf bucket's share to know when the quantile is unreliable. For alerting on latency, prefer ratio-of-buckets SLO expressions over a single quantile.
Confusing the scrape interval, evaluation interval, and rule for: duration, and expecting an alert to fire immediately after the condition becomes true.
Candidates test an expression that evaluates true and then wait for a page. The alert must first be pending for the full for: duration, evaluated at the evaluation interval, and the underlying samples only exist every scrape interval, so real detection latency is scrape + evaluation + for. They also set scrape_interval too long and lose resolution.
What to do instead
Know the chain: sample at scrape_interval, evaluate rules at evaluation_interval, require sustained breach for for:. Set scrape_interval shorter than evaluation_interval is not required, but keep for: a multiple of evaluation_interval, and size scrape_interval to the resolution your SLO needs. Verify state transitions on the /alerts page.
Assuming Prometheus pushes metrics, or that a target is up because the exporter process is running, without checking the up metric and the scrape health.
Candidates conflate the exporter being alive with Prometheus successfully scraping it. A target can be unreachable, return a non-200 status, or time out, and up == 0 is the only reliable signal. They also expect push semantics and are confused when a short-lived job's metrics never appear.
What to do instead
Alert on up{job="..."} == 0 and check the Targets page for scrape errors and last-scrape timestamps. Remember Prometheus pulls over HTTP; for short-lived jobs use the Pushgateway or a push-based collector, and never use the Pushgateway for long-running services. Confirm the exporter's /metrics endpoint responds before blaming PromQL.