Column projection is discarded: scans read and decode every column regardless of what the query needs

[ChronicallyJD]

Summary

A columnar scan reads and decodes every column of every row group it visits, regardless of how few columns the query needs. The projection list is computed correctly and then discarded. A query touching 1 column of a 12-column table costs the same I/O as one touching all 12.

This is the largest single remaining cost in the ungrouped-aggregate path measured in #289/#337, and it is the reason that residual is flat in the number of aggregated columns.

Measured

12-column, 4M-row table, 351 MB = 44,962 buffers total. PG17, max_parallel_workers_per_gather=0, buffers from EXPLAIN (ANALYZE, BUFFERS):

query	columns needed	buffers touched	% of table
count(*)	0	3	0%
sum(a)	1	45,094	100%
count(*), avg(b) WHERE a > 90	2	45,094	100%
sum of all 12 columns	12	45,092	100%

count(*) correctly costs nothing (the metadata path from #133). Everything else costs the whole relation.

Cross-check against table width — the same 2-column query, on tables differing only in how many columns they have:

table	total size	buffers touched	% of table	time
2 columns	59 MB	7,620	100%	149 ms
12 columns	351 MB	45,097	100%	893 ms

Both read exactly the whole relation. The 12-column table pays 5.9x the I/O for identical work; ~10/12 of those bytes are never used. Independent of the #337 batch fold — fold on and off both read 100%.

Cause

columnar_native_load_group, src/columnar_reader.c:933, reads the row group's bytes whole:

rs->nativeBuffer = palloc(rg->byteLength > 0 ? rg->byteLength : 1);
if (rg->byteLength > 0)
    ColumnarReadLogicalData(rs->rel, rg->fileOffset, rs->nativeBuffer, rg->byteLength);

projectedColumns is built in columnar_customscan.c:844, passed through ColumnarBeginRead (columnar_reader.c:280, :291), and copied into the read state at columnar_reader.c:334:

readState->projectedColumns = bms_copy(projectedColumns);

That assignment is the only occurrence of readState->projectedColumns in the tree. It is never read. The chunk-decode loop at columnar_reader.c:947 does not consult it either — it decodes every chunk in the group, skipping only out-of-range column indexes — so both the I/O and the decode are paid at full table width.

Why this is fixable locally

The per-column byte ranges are already recorded and already used. Each NativeColumnChunkMetadata carries pageOffset/pageLength, and the loader already indexes into the group buffer with them (columnar_reader.c:957):

base = rs->nativeBuffer + (cc->pageOffset - rg->fileOffset);

Each chunk is self-contained: its validity bitmap first, then its values. So reading only the needed chunks' ranges rather than one contiguous whole-group range is a change confined to the loader, with no on-disk format change. Chunks are ~1 MB at this shape, so page-alignment waste at range edges is negligible.

Expected payoff

Bounded by the ratio of projected width to total width. For a TSBS-shaped 12-column table with a 2-column query, that is up to ~6x less I/O, plus the decode of 10 columns nobody asked for. This is orthogonal to zone-map and bloom group skipping, which reduce which groups are visited, not what is read within a visited group.

Notes

• Correctness is not affected today; this is purely wasted work.
• Worth confirming whether the parallel scan path and the vectorized-aggregate paths all funnel through columnar_native_load_group (they appear to) so one fix covers them.
• Any fix needs a buffer-count assertion in the test suite, not a timing one: the measurement above is the natural regression test, and it fails loudly if projection silently stops applying.