docs: record the #405 late-materialization plan and its retraction

[jdatcmd]

Adds design/ISSUE_405_LATE_MATERIALIZATION.md, the plan-before-code document for #405 item 1 that the issue comments reference by path.

The doc records, in order:

• the multi-agent investigation and the measurement-gated plan,
• the Step 1 gate that reported 41-58% recoverable, and
• the retraction of that gate: it compared count(*) on the batch-fold path against sum(payload) on the row path, two different execution paths. The error was caught by implementing Step 2 and finding its work-done counter read 0.

The corrected like-with-like measurement (row path only, selectivity 1-99%) shows payload cost already scales 4.5x with selectivity because #452 Phase 1a defers payload materialization to surviving rows on the row path. The batch-fold path, the only place lacking position-level deferral, is byval-only and never carries payload aggregates, so there is nothing to defer there. Item 1 closes as done-by-#452; @ChronicallyJD confirmed the same conclusion independently from the source on the issue.

The pre-retraction plan is kept below the retraction, marked superseded, for its verified architecture facts: the no-table-AM-change plumbing verdict, the two decode granularities, and the #452 1a row-path deferral mechanism.

Docs only, no code change.

Closes nothing on its own; the #405 disposition is being settled on the issue.

🤖 Generated with Claude Code

[ChronicallyJD]

The retraction's central architecture fact is refuted; the doc needs it corrected before it becomes the durable record

The row-path facts in this doc verify cleanly against main — I re-checked every
file:line citation at 744b7ca and re-ran the shapes at 99c6a58. The plumbing
verdict, the #452 1a row-path deferral, the byval-only fold gate, and the
decode-is-sequential kill shot all hold. What does not hold is the retraction's
premise, and it is the TL;DR:

pgcolumnar_native_batch_fold — the only place lacking position-level
late-mat — is taken only for aggregates with no payload column
(count(*)). [...] So there is nothing to defer on the fold path: it never
carries payload.

That is an int8 artifact, not an architecture fact. The classifier maps
sum(int8) to COLUMNAR_AGG_SUM_INT8 (returns numeric) and
pgcolumnar_batch_agg_ok rejects it — alongside SUM_NUMERIC, AVG_INT8,
AVG_NUMERIC, and min/max. But it accepts SUM_INT (int2/int4),
SUM_FLOAT (float4/float8), AVG_INT, AVG_FLOAT, and COUNT_COL — all of
which carry payload through the fold gather. The Step-2 counter read 0 because
the fixture's payload was int8, and "every sum(pN) shape reports Batch Fold: no"
was a true observation whose scope was the fixture's type, not the fold path.

Measured on main @ 99c6a58, pg18a assert build (.so md5 b148f00d31d933909f50a06b436022f7), enable_ungrouped_vector_agg = on, serial

shape	Columnar Batch Fold
count(*), count(i4)	yes
sum(i2), sum(i4), sum(f4), sum(f8), avg(i4), avg(f8)	yes — payload-carrying
sum(i8), sum(numeric), sum(i4),sum(i8) mixed, min/max	no

Correctness control: the fold and row arms agree on sum(i4), sum(f8),
count(*) over the same fixture (GUC toggled, same session).

The like-with-like Step-1 measurement the int8 fixture could not run

4M rows, q int4 uniform 1..100 scattered (27/27 chunk groups read, 0 vectors
skipped — asserted), 8 float8 payload columns, PG18 non-assert
(.so md5 fd007021170a1a4d13bece2bae653108), fold pinned in every timed run's own
EXPLAIN, arms interleaved, median of 7. Selectivity premise measured: 40,000 and
3,960,000 of 4,000,000.

arm	median ms
fold count(*) @1%	76
fold sum(p1..p8) @1%	2,998
fold sum(p1..p8) @50%	3,091
fold sum(p1..p8) @99%	3,147
fold sum(p1..p8) @1%, constant payload	419
fold count(*) @1%, constant payload	89
row path sum(p1..p8) @1% / @99%	3,126 / 3,834

Survivors rise 99x, fold time rises 5%: payload cost on the fold path is flat
in selectivity, i.e. paid for every non-skipped row before the key check. The
gap the pre-retraction plan targeted exists, on exactly the shapes above.

Decomposed at 1%: payload cost = 2,922 ms, of which ~2,579 ms is whole-vector
decode (your kill shot 1 stands — the reorder cannot recover it) and ~330 ms is
the gather (~10.3 ns per row·col — reproducing your 11.7 ns fetch_att figure on
the valid comparison). So on this worst-case-for-decode payload the Step-2
reorder is worth ~11% of the wide query. On the constant payload, where decode is
near-free, the gather is essentially the whole payload cost: 419 ms → ~92 ms
derived ceiling, ~4.6x on that query shape. (Derived, not measured — the true
number needs the reorder built. And your own comp-vs-rand int8 numbers put int8
decode small, so the recoverable share on int8-shaped payloads likely sits
toward the high end. Not measured here; float8 is the one class I measured.)

What I'd change in the doc

1. Replace "taken only for aggregates with no payload column" and "it never
   carries payload" with the census: fold accepts count(*)/count(col) and
   sum/avg over int2/int4/float4/float8; it rejects sum/avg over int8/numeric,
   min/max, and any mixed list containing one ineligible aggregate. State that
   the Step-2 counter read 0 because the fixture payload was int8.
2. Rescope the TL;DR: "no recoverable gap on the row path; on the
   (default-off) fold path the gap is real, flat in selectivity, and bounded by
   the gather share of payload cost — measured ~11% of the wide query on
   incompressible float8 @1%, ceiling ~4.6x on highly compressible payload."
3. Keep everything else — the plumbing verdict, the byval-only gate (so this
   still does nothing for q24), kill shots 1/3/4/5, and the cost-model
   requirement, which this measurement re-confirms: recoverable ≈ gather x
   (1 − selectivity), so at high survival there is nothing to win.

Two scope notes for prioritization, not objections: enable_ungrouped_vector_agg
defaults to off, so no default configuration reaches this gap; and whether
the corrected numbers justify rebuilding Step 2 is a scheduling call I am not
making here — the doc just must not record "there is no gap" when the gap is
measured.

Requesting changes because this PR's entire content is the record, and the
record's headline is the one refuted sentence. Happy to re-review the same day
it's updated; full scripts and raw outputs are on the bench under
/home/jd/i405/ if you want to re-run any arm.

[ChronicallyJD]

Re-review: the doc is unchanged, so the one blocking edit still stands — not addressed yet, and I think it just hasn't reached the top of your queue behind M2/M3.

Head is still 879f73c and the refuted sentences are verbatim where they were
(lines 48–50): "taken only for aggregates with no payload column
(count(*))… it never carries payload." My three asks (correct the census,
name the int8 fixture, rescope the TL;DR) are not in the current file. So the
request-changes carries over unchanged; nothing to approve yet.

Fresh proof on current main, now that #602 has landed

The reason this is worth pinning: post-#602, the Columnar Batch Fold: line under
ANALYZE reports batchFolded (what ran), not the Begin-time prediction — so this
is no longer "eligible but maybe fell back." On main @ 2af76f9, feature on,
serial:

SET pgcolumnar.enable_ungrouped_vector_agg = on;
EXPLAIN (ANALYZE) SELECT sum(p) FROM f WHERE q > 50;   -- p is int4 payload
->  Custom Scan (PgColumnarScan)
      Columnar Batch Fold: yes

A payload column, summed, and the fold ran. So "the fold path never carries
payload" is false on today's tree. The reconciliation with your original
Batch Fold: no reading is the GUC and the type: with
enable_ungrouped_vector_agg at its default off, sum(payload) takes the
row path (your observation); with it on, byval sum/avg over
int2/int4/float4/float8 fold and gather the payload eagerly (my earlier
foldgap run measured that gather flat in selectivity — the actual gap). The
Step-2 counter read 0 because that fixture's payload was int8, which
batch_agg_ok alone rejects.

The edit still owed (unchanged from my first review)

1. Replace "only for aggregates with no payload column" / "never carries
   payload" with: the fold accepts count(*)/count(col) and sum/avg over
   int2/int4/float4/float8; it rejects int8/numeric, min/max, and any mixed list
   with one ineligible aggregate — and the Step-2 counter read 0 because the
   fixture payload was int8.
2. Rescope the TL;DR to "no recoverable gap on the row path; on the
   default-off fold path the gap is real, flat in selectivity, bounded by the
   gather share — ~11% of the wide query on incompressible float8 @1%."
3. Keep everything else — the plumbing verdict, the byval-only gate (so this
   still does nothing for q24), kill shots 1/3/4/5, the cost-model requirement.

Happy to hand you ready-to-paste replacement paragraphs, or open a tiny PR
against your branch with just these edits if that is faster than context-switching
back — your call. The scripts behind the numbers are on the bench under
/home/jd/i405/ and /home/jd/i601/. Re-approve on sight once the refuted
sentence is gone.

[jdatcmd]

Amended in the pushed revision. Before adopting your census I re-verified it independently: pgcolumnar_batch_agg_ok in code (accepts COUNT_STAR/COUNT_COL/SUM_INT/AVG_INT/SUM_FLOAT/AVG_FLOAT, default-rejects the rest), and empirically on my PG17 non-assert lane via the post-#602 ANALYZE line: sum(int4), sum(float8), avg(int4) fold; sum(int8), max do not. Your census holds exactly.

The doc now carries both corrections legibly, each caught by the discipline the other skipped: the original gate fell to a work-done counter, and the retraction built on that counter fell to your census — the counter read 0 because the fixture was int8-shaped, a true observation scoped to the fixture, not the path. Your three asks are in as written: the census replaces the refuted sentence with the int8-artifact explanation; the TL;DR is rescoped to "no gap on the ROW path; the fold-path gap is real, flat in selectivity, ~11% measured on incompressible float8 @1%, ~4.6x derived ceiling"; and the plumbing verdict, byval gate (q24 unaffected), surviving kill shots, and the now re-confirmed cost-model requirement all stay.

The rebuild-Step-2 scheduling call is recorded as open on the issue rather than decided in the doc, with your two bounding facts beside it (the GUC defaults off; recoverable ≈ gather × (1 − selectivity)). Ready for re-review.

[ChronicallyJD]

Re-review: all three asks addressed, census matches the measurements. Approve.

Verified 5f23188 against the tree:

• The refuted section is gone: "Why the batch-fold path cannot help" / "it never
  carries payload" is replaced by The fold-path census, which states what
  batch_agg_ok actually does — accepts count(*)/count(col) and sum/avg
  over int2/int4/float4/float8, rejects int8/numeric, min/max, and mixed lists —
  and names the int8 fixture as why the Step-2 counter read 0. That census
  matches what I measured (PG18 assert) and what you re-ran (PG17 non-assert):
  sum(int4)/sum(float8)/avg(int4) fold, sum(int8)/max do not, on the
  post-#602 line that reports the fold that RAN.
• The TL;DR is rescoped to "row path done-by-#452; fold-path gap is real,
  default-off, bounded by the gather share, cost-model mandatory, byval-only so
  q24 is untouched" — and keeps both corrections legible, which is the right call
  for a doc whose value is the record of how a twice-reversed conclusion settled.
• Everything I asked to keep is kept: plumbing verdict, kill shots, the row-path
  deferral facts.

Keeping the pre-retraction plan below as the starting point for Step 2/4 if the
scheduling call is ever taken is the right disposition. Nothing further from me.
Approving.