Prometheus histogram versus summary

WHAT THIS TESTS Whether you understand the practical Prometheus consequence of computing quantiles client-side versus query-side, especially across many servers.

A GOOD ANSWER COVERS A histogram samples observations into configurable cumulative buckets and exposes three things: the per-bucket counts as metric_bucket with a le label, the total metric_sum, and metric_count. Quantiles are not stored; you compute them at query time with histogram_quantile, and because bucket counts are additive you can sum buckets across the whole fleet first and then derive a correct fleet-wide quantile, bounded in accuracy by bucket width. A summary computes a fixed set of quantiles, like 0.5, 0.9, 0.99, locally on each instance over a sliding time window, and exposes them directly plus sum and count. Those quantiles are accurate for one instance but cannot be aggregated: you cannot validly combine per-instance p99 values. Summaries also lock in which quantiles exist when you write the code.

COMMON WRONG ANSWERS Saying you can average summary quantiles to get a fleet quantile. Claiming histograms give exact quantiles regardless of buckets. Ignoring that summaries do more work on the client and histograms can be heavier on series count due to buckets.

LIKELY FOLLOW-UPS How do you choose bucket boundaries? What do native or exponential histograms change? How does bucket count interact with label cardinality?

ONE CONCRETE EXAMPLE To measure p99 request latency across 30 servers, you instrument with a histogram http_request_duration_seconds. The query histogram_quantile(0.99, sum by (le) (rate(http_request_duration_seconds_bucket[5m]))) gives a correct fleet-wide p99. Had you used a summary, each server would report its own p99 and you would have no statistically valid way to combine them, so histograms are the right choice for fleet latency.