Prometheus Certified Associate: Instrumentation and Exporters

Instrumentation and Exporters tests whether you can expose useful metrics without breaking cardinality or semantics. It covers client libraries, metric types, naming, and the exporter pattern.

Metric types and conventions

• Counter: monotonically increasing, conventionally _total; resets on restart. Use for requests, errors, bytes.
• Gauge: current value that can rise or fall, such as queue depth or temperature.
• Histogram: cumulative buckets plus _sum and _count; aggregatable across instances if buckets match. histogram_quantile estimates quantiles.
• Summary: client-side quantiles plus _sum and _count; quantiles are not aggregatable across instances.
• Use base units: seconds and bytes. Use labels for bounded dimensions such as method and status, never for unbounded IDs.
• Exporters translate third-party metrics: node_exporter for hosts, blackbox_exporter for probes, mysqld_exporter for MySQL. Pushgateway accepts pushed metrics for batch jobs.

Worked example

Instrument latency with buckets 0.1, 0.5, 1, 2.5, 5, +Inf. A request taking 0.7s increments le='1', le='2.5', le='5', and le='+Inf', but not le='0.5'. Query p90 with histogram_quantile(0.9, sum by (le) (rate(request_duration_seconds_bucket[5m]))). If you instead used a summary, each instance's 0.9 quantile could not be combined into a fleet-wide p90.

Traps

• Creating a new metric name per user or request instead of using labels.
• Using a gauge for a total that only rises, or a counter for queue depth.
• Assuming summaries aggregate across instances.
• Forgetting that exporters expose their own metrics too, which can add cardinality.