Instrumentation (Automatic and Manual)
Automatic (zero-code) instrumentation injects telemetry without source changes, typically via an agent, language-specific init hooks (e.g., -javaagent, auto-instrumentation packages, eBPF), and produces spans for common libraries (HTTP clients, DB drivers, frameworks). Manual instrumentation uses the API directly to create spans, metrics, and log records for business logic that auto-instrumentation cannot see.
Candidates must know the trade-offs. Auto-instrumentation is fast to adopt and consistent but is library-specific and gives limited business context. Manual instrumentation is precise but requires code and discipline. The two compose: manual spans nest under auto-generated ones because they share context.
Key manual patterns: obtain a Tracer from a TracerProvider and a Meter from a MeterProvider; start a span, set attributes, record errors via status and recordException, and always end the span (use try/finally or scope). Use span events for point-in-time occurrences and attributes for dimensions. Avoid high-cardinality attributes (user IDs, raw URLs) on metrics - they explode series counts.
Traps: forgetting to end a span (it never exports); creating a new Tracer per request (use a long-lived one); using setAttribute with non-primitive types; assuming auto-instrumentation captures custom business spans (it won't). Also, manual instrumentation should depend on the API, not the SDK.
Rule of thumb: one Tracer/Meter per instrumentation scope, created once and reused.
Worked example: A checkout service shows HTTP spans but no "validatePayment" step. Fix: add a manual span around validatePayment, set an error status on failure, and end it in a finally.