tezvyn:

How does columnar storage speed up analytics?

AI-drafted, machine-checkedSource: interviewintermediate
WHAT IT TESTS

physical storage layout versus query type.

OUTLINE

columnar stores each column contiguously, so aggregations read only needed columns, scan far less data, and compress better with vectorized execution.

WHAT THIS TESTS This checks whether you understand how physical data layout interacts with query shape, a key concept behind analytical databases and formats like Parquet.

A GOOD ANSWER COVERS Row-based storage keeps all fields of a record together on disk, which is ideal when you read or write whole rows, as OLTP does. Columnar storage instead keeps all values of a single column together contiguously. For analytical aggregations such as summing or averaging one column over millions of rows, the engine reads only that column's data and skips every other column entirely, slashing the amount of data scanned from disk. Because a column holds values of one type and often with low cardinality or sorted order, it compresses dramatically better using techniques like run-length, dictionary, and delta encoding, which further reduces I/O. Contiguous same-type values also enable vectorized, SIMD-style processing where the CPU operates on many values per instruction. The cost is that assembling or modifying an entire row means touching many separate column segments, making single-row inserts, updates, and lookups expensive, which is why columnar suits read-heavy analytics, not transactional workloads.

COMMON WRONG ANSWERS Saying columnar is just generally faster, ignoring that it is slow for row-at-a-time OLTP. Attributing the speedup only to compression while missing the bigger win of reading fewer columns. Forgetting vectorized execution. Believing columnar avoids reading any unneeded rows; it still scans needed columns fully unless additional pruning like min-max indexes applies.

LIKELY FOLLOW-UPS Why does columnar compress better, and which encodings help? How do formats like Parquet or ORC implement this? What is a hybrid like row groups or PAX? How do min-max statistics enable partition or chunk pruning?

ONE CONCRETE EXAMPLE A fact table has fifty columns and you compute the average order amount over a billion rows. A columnar engine reads only the amount column, perhaps a few gigabytes compressed, while a row store would read all fifty columns, hundreds of gigabytes, to extract one. The columnar scan finishes far faster with far less I/O.

Read the original → en.wikipedia.org

Get five bites like this every day.

Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.