Distributed Tracing: Following a Request Across Microservices

Why it exists

In a monolithic application, a single stack trace can pinpoint an error. In a microservices architecture, a single user request might hop between a dozen services. If it's slow or fails, you're left trying to correlate timestamps across a dozen different log files. Distributed tracing was created to solve this chaos.

The mental model

Think of it like a package delivery with a tracking number. The package is your request, and the tracking number is a unique 'Trace ID'. At every stop—the web server, the authentication service, the database—the request's journey is scanned. All these scans are sent to a central system, which reconstructs the package's full journey, showing exactly where it went and how long it spent at each stop.

How it works

When a request first enters your system, it's assigned a unique Trace ID. The work done by that first service is a 'span'. When it calls another service, it passes the Trace ID along in the request headers—a process called context propagation. Each subsequent service creates its own spans, all linked by the same Trace ID. An observability backend collects all these spans and assembles them into a single, chronological waterfall graph, showing the entire request lifecycle.

When to use it

It is indispensable in any distributed system, especially microservices. Use it to diagnose latency bottlenecks ('Why did this API call take 2 seconds?'), find the root cause of errors ('Which downstream service failed first?'), and understand complex dependencies between services.

When not to use it

For simple, monolithic applications, it can be overkill; traditional logging and metrics are often sufficient. Tracing also introduces a small amount of overhead for instrumentation and data collection, so it requires careful implementation in extremely high-throughput, low-latency systems.

One canonical example

OpenTelemetry provides a vendor-neutral standard for tracing. You can use its APIs and SDKs to instrument your code once. Then, you can send that telemetry data to any backend, like Jaeger, Prometheus, or a commercial vendor, using the OpenTelemetry Collector. This prevents vendor lock-in and allows you to switch observability tools without rewriting your application's instrumentation.