Application Instrumentation with Datadog
Application instrumentation with Datadog
This domain tests how you enable tracing in an application: installing a Datadog tracing library, applying Unified Service Tagging, using automatic and manual instrumentation, propagating context, and correlating logs. Questions often hinge on which setting belongs in the tracer versus the Agent.
Automatic versus manual instrumentation
Datadog tracing libraries exist for Python, Node.js, Java, .NET, Go, Ruby, PHP, and C++. Automatic instrumentation patches common frameworks and clients: web frameworks, HTTP clients, database drivers, cache clients, and message queues. Manual instrumentation covers business logic the library cannot infer. In Python you might use tracer.trace() or tracer.start_span(); in Java, GlobalTracer.get().buildSpan(...); in Node.js, tracer.trace(...). A manual span should get a meaningful resource name, a span type such as custom or business, and low-cardinality tags.
Unified Service Tagging and key settings
Set DD_SERVICE, DD_ENV, and DD_VERSION consistently across every service. These three tags join traces, logs, deployment events, and the Service Map. Other common settings include DD_AGENT_HOST and DD_TRACE_AGENT_PORT for reaching the Agent, DD_TRACE_ENABLED to turn tracing on or off, DD_TRACE_SAMPLE_RATE for head-based sampling, DD_LOGS_INJECTION to add trace IDs to logs, and DD_TRACE_PROPAGATION_STYLE_INJECT and DD_TRACE_PROPAGATION_STYLE_EXTRACT to choose Datadog, W3C, or B3 propagation. Tag spans with error information using error, error.msg, error.type, and error.stack so Error Tracking can group failures.
Worked example
A Python Flask service named checkout calls Redis and computes tax. Install ddtrace, run the app with ddtrace-run, and set DD_SERVICE=checkout, DD_ENV=prod, DD_VERSION=1.4.2, and DD_LOGS_INJECTION=true. Flask and Redis spans appear automatically. Wrap the tax calculation in a manual span with resource calculate_tax and tag customer_tier:gold, not order_id, which would create excessive cardinality. Verify in the Trace Explorer that the manual span is a child of the Flask request span and that logs contain trace_id and span_id.
Traps
Leaving DD_ENV unset produces env:none and breaks filtering and Service Map comparisons. Setting a 100% head sampling rate in high-volume production is costly. Tagging unique IDs on spans creates cardinality problems. Context does not automatically cross threads, thread pools, or some async boundaries without explicit propagation. A tracer version incompatible with the Agent can silently drop traces.