Prometheus Fundamentals
Prometheus Fundamentals covers the server data model and lifecycle: how targets are discovered and scraped, how samples become time series, and how storage, remote write, and federation fit together. The exam favors concrete configuration and data-model reasoning.
Data model and scrape lifecycle
A time series is uniquely identified by its metric name plus all label pairs. Labels are dimensions; the metric name is not a label. Prometheus pulls metrics over HTTP from /metrics at scrape_interval, applies scrape_timeout, and supports static configs, file SD, DNS SD, and cloud discovery. honor_labels, metric_relabel_configs, and relabel_configs control target and sample labels. Samples are appended to a local TSDB in 2-hour blocks; retention is set by time and/or size. Remote write ships samples elsewhere; federation lets a parent scrape selected series from children, not replace alerting.
Worked example
A metric http_requests_total{method,status,instance,path} has 5 methods x 10 statuses x 100 instances x 50 paths = 250,000 series. At a 15-second scrape, that is 20 samples per minute per series, or 5 million samples per minute across the set, before indexing overhead. Removing path or aggregating at the exporter cuts cardinality dramatically.
Traps
- Assuming Pushgateway is for long-running services; it is for short-lived batch jobs and does not expire groups automatically.
- Forgetting that counters reset when a process restarts; use
rateorincrease, not raw deltas. - Scraping too frequently without adjusting retention and disk.
- Using federation as a global long-term store.