Head-based vs tail-based trace sampling

WHAT THIS TESTS This probes whether you understand the timing of the sampling decision and can reason about cost versus signal quality in a tracing pipeline.

A GOOD ANSWER COVERS Head-based sampling makes the keep-or-drop decision at the very beginning of a trace, usually at the first service, before anything about the request outcome is known. It is typically a fixed probability or rate limit propagated down the trace via context, so all services honor the same decision. It is cheap, stateless, and predictable, but it is blind to whether the request ultimately errored or was slow. Tail-based sampling defers the decision until the entire trace has finished, buffering all spans in a collector and then applying policies such as keep all traces containing an error, keep all traces over a latency threshold, and sample a small percentage of the normal ones. This yields far richer signal but requires the collector to hold spans in memory until the trace completes, adds latency, and must handle spans arriving out of order across services.

COMMON WRONG ANSWERS Claiming head-based sampling can preferentially retain error or slow traces; it cannot, because the outcome is unknown when the decision is made. Another error is ignoring the statefulness and memory pressure tail-based sampling introduces.

LIKELY FOLLOW-UPS How does the collector know a trace is complete? How do you bound memory? Can you combine both strategies? How does sampling interact with metrics derived from traces?

ONE CONCRETE EXAMPLE A high-traffic API serves a million requests an hour, of which only a tiny fraction error or breach latency. With ten percent head-based sampling you would capture roughly ten percent of those rare failures and likely miss most incidents entirely. With tail-based sampling you configure policies to keep one hundred percent of error traces and all traces exceeding five hundred milliseconds, while sampling one percent of healthy traces. Now every incident is fully captured for debugging, storage stays bounded because failures are rare, and the on-call engineer always has the relevant trace, justifying the added collector complexity.