TSM-Tree vs LSM-Tree storage engines

WHAT THIS TESTS This probes deep storage-engine knowledge: how a time-series engine adapts the LSM idea to the unique shape of time-series data.

LSM-TREE BASICS A log-structured merge-tree absorbs writes into an in-memory memtable backed by a write-ahead log, then flushes the sorted memtable to an immutable on-disk file, an SSTable. Background compaction merges SSTables to remove overwrites and bound read amplification. This design turns random writes into sequential ones, giving excellent write throughput, which is why it backs Cassandra, RocksDB, and LevelDB. Reads may consult several SSTables plus the memtable, using bloom filters to skip files.

TSM-TREE The Time-Structured Merge tree in InfluxDB is an LSM derivative specialized for time-series. Incoming points go to a cache and a WAL, then flush to immutable TSM files. Crucially, data is organized by series key, then sorted by time, and stored column-wise: timestamps in one compressed block, values in another. Because timestamps arrive in near-monotonic order and adjacent values are similar, it applies type-aware encodings, delta-of-delta on timestamps, run-length and XOR-style encoding on values, achieving very high compression.

WHY IT IS MORE EFFICIENT FOR TIME-SERIES Writes are mostly ordered appends to the latest time range, so flushes and compaction stay cheap and rarely rewrite old data. A time-range scan for one series reads contiguous, densely compressed blocks rather than scattered key-value pairs, minimizing I/O. Columnar layout lets aggregations read only the value column. A general LSM stores opaque key-value pairs and cannot exploit time ordering or per-type compression as aggressively.

LIKELY FOLLOW-UPS How does delta-of-delta encoding work. What is high-cardinality and why does it hurt. How does compaction differ. When would a plain LSM suffice.

ONE CONCRETE EXAMPLE Storing CPU metrics every second, a TSM file holds one series' timestamps delta-of-delta encoded and values XOR-compressed in adjacent blocks. A query for the last hour reads a few contiguous compressed blocks, whereas a generic LSM would scan many interleaved key-value entries.