Vectorized decompression + aggregation for full-scan aggregates (~4× behind TimescaleDB)

[ChronicallyJD]

Vectorized decompression + aggregation (full-scan aggregates ~4× behind TimescaleDB)

Motivation

In the 100M-row TSBS-cpu benchmark (PG18), even on full-scan aggregates where pgColumnar
reads less data than TimescaleDB (2.67 GB vs 7.79 GB on disk), TimescaleDB is ~4×
faster:

query	pgColumnar	TimescaleDB	heap
q4 — avg/hour/host, 12 h	11,247 ms	2,648 ms	13,009 ms
q5 — avg of 10 metrics/hour/host	22,131 ms	5,747 ms	24,518 ms
q6 — full scan + filter	32,470 ms	6,131 ms	10,304 ms
q8 — top-20 host-hours	21,608 ms	4,637 ms	15,315 ms

(serial warm median.) pgColumnar beats heap on the wide aggregates (q4/q5) — the columnar
projection pays off — but is well behind TimescaleDB's vectorized execution.

Root cause

TimescaleDB decompresses its columnstore in batches and aggregates over those batches with
vectorized operators. pgColumnar's scan decodes each vector but then feeds rows into
PostgreSQL's row-wise executor for aggregation, paying per-row ExecAgg overhead over
100M rows. Correct, but the per-row path is the bottleneck once the data is already in
column-major form.

Proposed direction (cleanroom, larger effort)

Push aggregation down into the columnar scan and operate on decoded column batches:

• decode into column-major batches (already available inside ColumnarScan) and run
  batch aggregate kernels (sum/avg/min/max/count, filter predicates) over a
  vector at a time instead of row-at-a-time;
• evaluate the WHERE predicate on the batch to produce a selection vector before
  aggregating (helps q6);
• structure kernels to be SIMD-friendly (contiguous typed arrays, no per-row function-call
  dispatch);
• integrate with grouped aggregation (GROUP BY date_trunc(...), hostname) via a
  vectorized hash-aggregate, or at minimum vectorize the per-group accumulation.

Reference the public literature and systems concepts only (MonetDB/X100 vectorized
execution, DuckDB vectorized operators, the Gorilla/columnar aggregation papers); the
implementation must be ours alone.

Impact

Closes the ~4× full-scan gap to TimescaleDB and compounds with parallelism (pgColumnar
already scales q4/q5 across workers). Lower priority than the segmentby-clustering gap,
which is a larger multiplier on the selective queries, but this is the next lever on the
full-scan analytic workload.

Related

• Segmentby-clustering gap (data-skipping) — the larger query lever; this one is
  orthogonal (reduces cost per scanned row, not rows scanned).
• Index-scan row fetch re-decodes a full chunk per row (~30 ms/row): selective queries 1000× slower or time out #287 — fetch path (index scans); separate from the scan-aggregate path.
• Full data + method: bench-tsdb/BENCHMARK_RESULTS.md.