Downsampling: Trading Granularity for Speed in Time Series Data

Why it exists

Storing high-resolution metrics forever is expensive and makes querying over long time ranges, like a full year, incredibly slow. Downsampling solves this by creating lower-resolution, summarized copies of old data, making historical analysis fast and cost-effective.

The mental model

Think of downsampling like a map with different zoom levels. When viewing a single street, you need high detail. When viewing a whole country, you only need major cities. Downsampling creates these "zoomed-out" views of your metrics, trading fine-grained detail for a faster, broader overview of historical data.

How it works

A dedicated process, like the Thanos Compactor, scans for blocks of data past a certain age. It then creates new, downsampled blocks by aggregating the raw data points. For example, it might take all the raw points from a 5-minute window and compute their average, sum, or max to create a single new point. This process is often tiered; as data gets even older, it can be downsampled again into an even coarser resolution, like 1-hour aggregates.

When to use it

Use downsampling when you need to retain metrics for months or years but don't need second-by-second precision for old data. It is critical for systems that power historical dashboards, capacity planning reports, and long-term trend analysis, ensuring queries remain responsive.

When not to use it

Avoid downsampling if you have a strict requirement to retain raw, high-fidelity data for its entire lifecycle. The process is also not for fresh data; there is an intentional delay because you need a large window of raw data to create an accurate, aggregated sample. For example, you might need 40 hours of data before you can start creating 5-minute downsamples.

One canonical example

In the Thanos monitoring system, the Compactor component handles downsampling. A typical configuration creates 5-minute resolution blocks for data older than 40 hours. Then, for data older than 10 days, it creates even coarser 1-hour resolution blocks. This tiered approach keeps recent queries highly detailed while making long-range historical queries fast.