Signals: Traces, Metrics, Logs, and Baggage
Each signal has distinct semantics and pitfalls.
Traces model request flow. A span has a kind (server, client, producer, consumer, internal) that describes its role, not who created it. Spans form parent/child relationships; a root span has no parent. Watch for questions conflating span kind with span status (status is unset/ok/error).
Metrics come in instrument types: Counter (monotonic sum), UpDownCounter (non-monotonic sum), Histogram (distribution, e.g., latency), and Gauge (current value, observable). Know the difference between synchronous instruments (recorded inline) and asynchronous/observable instruments (callback-based). A common trap: a Histogram is for distributions and yields count/sum/buckets - not a single average. Also understand aggregation temporality (cumulative vs. delta) and exemplars, which link a metric data point to a trace.
Logs in OTel are structured records with a severity, body, attributes, and optional trace context. The Log Bridge maps existing logging frameworks into OTel. A frequent misconception: OTel does not replace your logging library; it standardizes the record model and can correlate logs with traces via trace/span IDs.
Baggage is key/value data propagated across service boundaries alongside context. It is not automatically added to spans or metrics - you must read it explicitly. It is also not a security boundary; do not put secrets in baggage. Baggage propagates; span attributes do not.
Rule of thumb: Baggage travels with the request; attributes stay on the span.
Worked example: To correlate a metric spike with a specific slow request, use exemplars on a Histogram - they attach a trace ID to a bucket sample.