Vectorized query execution and its speedups

WHY IT EXISTS The classic Volcano or iterator model calls next() once per tuple through each operator. That means a virtual function call and interpretation overhead for every single row at every operator, which dominates CPU time on large analytical scans where the per-row work is small. Vectorized execution exists to amortize that overhead.

THE MENTAL MODEL Instead of pulling one row at a time, each operator's next() returns a batch, a vector of many values, usually organized by column. Operators work over whole batches: a filter evaluates the predicate across the entire vector, an aggregation accumulates over the batch, and so on.

HOW IT WORKS AND WHY IT IS FASTER Processing a batch per call spreads the function-call and dispatch cost over hundreds or thousands of values, so per-tuple overhead nearly vanishes. The inner loops are tight and operate on contiguous column data, which fits CPU cache lines and prefetches well, raising cache locality. Predictable straight-line loops improve branch prediction, and crucially the compiler and CPU can apply SIMD, single-instruction-multiple-data, to compute several values at once. Together these dramatically increase instructions retired per cycle.

WHEN IT MATTERS AND DISTINCTIONS It mainly helps scan-heavy, aggregation-heavy analytical (OLAP) workloads. It is distinct from, though complementary to, columnar storage, query parallelism, and JIT query compilation; vectorization is about batch granularity, not multiple cores.

LIKELY FOLLOW-UPS How does vectorization compare to compiled query execution, why does it pair naturally with columnar layout, and what batch size balances overhead against cache footprint.

ONE CONCRETE EXAMPLE Summing a billion-row column in the Volcano model does a billion next() calls. A vectorized engine pulls vectors of, say, a few thousand values and sums each with a SIMD loop, cutting call overhead by orders of magnitude and running several additions per CPU instruction, which is why engines like ClickHouse process such queries far faster.