Skip to content

feat: fold-path payload deferral with an adaptive gate (#405 step 2) #617

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jdatcmd merged 1 commit into from
Aug 13, 2026
Merged

feat: fold-path payload deferral with an adaptive gate (#405 step 2) #617

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions design/ISSUE_405_LATE_MATERIALIZATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -310,3 +310,30 @@ position-level late-mat, but it is byval-only and the residual is a decoded-buff
load, so the win is bounded by a wall-clock measurement not yet taken." The row
path — where the expensive varlena decode lives — already defers. The measurement
in Step 1 decides whether even the fold slice is worth it; the honest prior is no.

---

## Step 2 rebuild (scheduled 2026-08-13): the implementation shape, one amendment

The owner scheduled the rebuild (issue comment, 2026-08-13). Steps 2 and 3
are implemented as this document planned: the gather loop defers non-key
`fetch_att` until the scan-key check passes, payload cursors advance without
reading on failing rows (the consume-slot-do-not-read contract vecSkipped and
deleted rows already follow), and the work-done counter increments at the
stable post-key site with its own label, `Columnar Fold Payload Loads`, so no
EXPLAIN line means two different quantities on two plans (Step 3 by
construction).

**Amendment to Step 4**: the cost gate is RUNTIME-ADAPTIVE, not the planner
plumbing this plan sketched. Per group, deferral is enabled iff the survival
fraction observed over all previous groups is at most one half; the first
group is optimistic. Rationale for the deviation, recorded per house rule:
the planner route requires extending the agg node's positional
custom_private and its list_length discriminator plus an input_rel width
computation, all to consume a planner selectivity estimate that the executor
can simply measure; the adaptive gate needs no statistics, self-corrects on
data the planner mispredicts, costs one comparison per group boundary (the
#289 no-per-group-precompute guard holds), and is directly observable as
`Columnar Fold Deferred Groups: X of Y`. The reviewer may push back to the
planner design; the suite's arms bind the BEHAVIOUR (deferral on selective
data, eager fallback on high survival), not the mechanism.
125 changes: 121 additions & 4 deletions src/columnar_vector.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -582,6 +582,18 @@ typedef struct PgColumnarAggScanState
bool batchEligible;
bool batchFolded;

/*
* Fold-path payload deferral (#405 step 2). foldPayloadLoads counts
* fetch_att materializations of non-key columns, incremented at the
* stable post-key site when deferring and at the eager gather otherwise,
* so the counter measures the work either way and the deferral's saving
* is their difference. foldGroupsDeferred/foldGroupsTotal expose the
* runtime-adaptive gate (defer unless observed survival exceeds half).
*/
uint64 foldPayloadLoads;
int foldGroupsDeferred;
int foldGroupsTotal;

/*
* Parallel batch fold (#289 phase 5/6): a partial node emits its per-worker
* transition state instead of the finalized value, and its reader claims
Expand Down Expand Up @@ -3124,8 +3136,13 @@ pgcolumnar_native_batch_fold(PgColumnarAggScanState *state, Relation rel,
int16 *cattlen = (int16 *) palloc0(sizeof(int16) * natts);
uint64 *cpresent = (uint64 *) palloc0(sizeof(uint64) * natts);
bool *cneeded = (bool *) palloc0(sizeof(bool) * natts);
bool *ciskey = (bool *) palloc0(sizeof(bool) * natts);
Datum *cval = (Datum *) palloc0(sizeof(Datum) * natts);
bool *cisnull = (bool *) palloc0(sizeof(bool) * natts);
int npayload = 0;
int64 candRows = 0; /* rows that reached the key check */
int64 survRows = 0; /* rows that passed it */
bool deferOn = false;

if (!pgcolumnar_batch_shape_eligible(state, tupdesc, &keys, &nkeys))
return false;
Expand All @@ -3135,6 +3152,27 @@ pgcolumnar_native_batch_fold(PgColumnarAggScanState *state, Relation rel,
if (col >= 0 && col < natts)
cneeded[col] = true;

/*
* Payload deferral (#405 step 2): key columns are the ones the scan keys
* read; every other needed column is payload, whose fetch_att can wait
* until the key check passes. ciskey is the split; npayload > 0 is the
* only case with anything to defer.
*/
{
int kk;

for (kk = 0; kk < nkeys; kk++)
{
int ka = keys[kk].sk_attno - 1;

if (ka >= 0 && ka < natts)
ciskey[ka] = true;
}
for (col = 0; col < natts; col++)
if (cneeded[col] && !ciskey[col])
npayload++;
}

/*
* Push the scan keys so the reader prunes whole row groups its zone maps rule
* out (#349). The fold walks every surviving group in full and still rechecks
Expand Down Expand Up @@ -3183,6 +3221,19 @@ pgcolumnar_native_batch_fold(PgColumnarAggScanState *state, Relation rel,
PgColumnarReadFoldGroupInfo(rs, &nrows, &dmask, &dlen,
&skipVec, &decodeSkipped, &vecStart, &vcount);

/*
* The adaptive gate (#405, amended step 4): defer this group's payload
* unless the survival observed so far exceeds half, in which case the
* two-phase gather would refetch nearly everything and eager is
* cheaper. The first group is optimistic. One comparison per group:
* the #289 no-per-group-precompute guard holds.
*/
deferOn = (npayload > 0 &&
(candRows == 0 || survRows * 2 <= candRows));
state->foldGroupsTotal++;
if (deferOn)
state->foldGroupsDeferred++;

/*
* This loop now honours skipVec, and it has to (#512, #452 phase 1b-i).
* Decode no longer produces the vectors the zone maps ruled out, so the
Expand Down Expand Up @@ -3273,10 +3324,15 @@ pgcolumnar_native_batch_fold(PgColumnarAggScanState *state, Relation rel,
vecSkipped = (curVec < vcount && skipVec[curVec]);
}

/* gather needed values at each column's present index; advance it */
/*
* Phase 1: gather the scan-key columns (and, when this group is
* eager, everything). Deferred payload columns are not touched
* here at all -- not even their cursors -- so the failing-row
* path below can consume their slots without a fetch (#405).
*/
for (col = 0; col < natts; col++)
{
if (!cneeded[col])
if (!cneeded[col] || (deferOn && !ciskey[col]))
continue;
if ((cvalidity[col][r >> 3] >> (r & 7)) & 1)
{
Expand All @@ -3289,6 +3345,8 @@ pgcolumnar_native_batch_fold(PgColumnarAggScanState *state, Relation rel,
cval[col] = fetch_att(cpacked[col] + cpresent[col] * cattlen[col],
true, cattlen[col]);
cisnull[col] = false;
if (!ciskey[col])
state->foldPayloadLoads++;
}
cpresent[col]++;
}
Expand All @@ -3297,12 +3355,28 @@ pgcolumnar_native_batch_fold(PgColumnarAggScanState *state, Relation rel,
}

if (vecSkipped)
continue; /* value slots already consumed above */
{
/* consume deferred payload slots; never read them (#405) */
if (deferOn)
for (col = 0; col < natts; col++)
if (cneeded[col] && !ciskey[col] &&
((cvalidity[col][r >> 3] >> (r & 7)) & 1))
cpresent[col]++;
continue;
}

del = (dmask != NULL && (r >> 3) < dlen &&
(dmask[r >> 3] & (1 << (r & 7))) != 0);
if (del)
continue; /* value slots already consumed above */
{
if (deferOn)
for (col = 0; col < natts; col++)
if (cneeded[col] && !ciskey[col] &&
((cvalidity[col][r >> 3] >> (r & 7)) & 1))
cpresent[col]++;
continue;
}
candRows++;

for (k = 0; k < nkeys; k++)
{
Expand All @@ -3319,7 +3393,38 @@ pgcolumnar_native_batch_fold(PgColumnarAggScanState *state, Relation rel,
}
}
if (!pass)
{
if (deferOn)
for (col = 0; col < natts; col++)
if (cneeded[col] && !ciskey[col] &&
((cvalidity[col][r >> 3] >> (r & 7)) & 1))
cpresent[col]++;
continue;
}
survRows++;

/*
* Phase 2 (#405): the row survived, so NOW materialize its
* deferred payload. This is the stable post-key counter site the
* plan required: on a deferred group the counter equals surviving
* rows times fetched payload values, never candidates.
*/
if (deferOn)
for (col = 0; col < natts; col++)
{
if (!cneeded[col] || ciskey[col])
continue;
if ((cvalidity[col][r >> 3] >> (r & 7)) & 1)
{
cval[col] = fetch_att(cpacked[col] + cpresent[col] * cattlen[col],
true, cattlen[col]);
cisnull[col] = false;
state->foldPayloadLoads++;
cpresent[col]++;
}
else
cisnull[col] = true;
}

for (a = 0; a < state->naggs; a++)
{
Expand Down Expand Up @@ -3585,6 +3690,9 @@ PgColumnarReScanAggScan(CustomScanState *node)
state->done = false;
state->haveStats = false;
state->batchFolded = false;
state->foldPayloadLoads = 0;
state->foldGroupsDeferred = 0;
state->foldGroupsTotal = 0;
MemoryContextReset(state->resultContext);
for (a = 0; a < state->naggs; a++)
{
Expand Down Expand Up @@ -3646,6 +3754,15 @@ PgColumnarExplainAggScan(CustomScanState *node, List *ancestors, ExplainState *e
*/
PgColumnarGroupStats gs;

if (state->batchFolded)
{
ExplainPropertyInteger("Columnar Fold Payload Loads", NULL,
(int64) state->foldPayloadLoads, es);
ExplainPropertyText("Columnar Fold Deferred Groups",
psprintf("%d of %d", state->foldGroupsDeferred,
state->foldGroupsTotal), es);
}

gs.usableSkipPredicates = state->usablePreds;
gs.groupsTotal = state->groupsTotal;
gs.groupsRead = state->groupsRead;
Expand Down
107 changes: 107 additions & 0 deletions test/native_fold_deferral.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
#!/usr/bin/env bash
#
# pgColumnar #405 step 2: fold-path payload deferral, with the work MEASURED.
#
# On the batch-fold path, non-key columns used to be fetch_att'd for every
# non-skipped row BEFORE the scan-key check; the corrected #405 record measured
# that cost flat in selectivity. The fold now defers payload materialization
# until a row passes its keys, gated adaptively per group: defer unless the
# survival observed so far exceeds half (the first group is optimistic), so
# high-survival shapes keep the eager single pass that is cheaper for them.
#
# The counter is the plan's stable post-key site: on a deferred group,
# Columnar Fold Payload Loads equals surviving rows x payload values fetched -
# never candidates - which is the work-done identity the original Step 2
# failed to have and the retraction was caught by. The gate is observable as
# Columnar Fold Deferred Groups: X of Y.
#
# Oracles: the row path (vector agg off) and a heap mirror; the fold node is
# pinned via the post-#602 ANALYZE line, which reports the fold that RAN.
#
# Usage: test/native_fold_deferral.sh [PG_CONFIG]
# Written fresh for pgColumnar.

set -uo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/lib.sh"
pgc_setup "${1:-/usr/local/pg17/bin/pg_config}"

GUC="SET pgcolumnar.enable_ungrouped_vector_agg = on;"
NP=4 # payload columns
ROWS=120000

sq() { # scalar under this suite's GUC; tail -1 drops the SET tag
env PATH="$PGC_BINDIR:$PATH" psql -h 127.0.0.1 -p "$PGC_PORT" -U postgres \
-d "$PGC_DB" -At -c "$GUC $1" 2>/dev/null | tail -1
}
explain_line() { # explain_line <label> <query>
env PATH="$PGC_BINDIR:$PATH" psql -h 127.0.0.1 -p "$PGC_PORT" -U postgres \
-d "$PGC_DB" -At -c "$GUC EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF) $2" 2>/dev/null \
| grep "$1" | head -1 | sed 's/^ *//'
}

psql_run "CREATE TABLE fd_t (q int4, p1 float8, p2 float8, p3 float8, p4 float8) USING pgcolumnar;"
psql_run "SELECT pgcolumnar.set_options('fd_t', stripe_row_limit => 20000);"
psql_run "INSERT INTO fd_t SELECT 1 + g % 100, g, g*2, g*3, g*4 FROM generate_series(1,$ROWS) g;"
psql_run "CREATE TABLE fd_heap AS SELECT * FROM fd_t;"

# plain aggregate tlist entries: an expression OVER aggregates disqualifies
# the vectorized path entirely (the #602 tlist trap) and would row-path this
QSEL="SELECT sum(p1), sum(p2), sum(p3), sum(p4) FROM fd_t WHERE q <= 1"
QHI="SELECT sum(p1), sum(p2), sum(p3), sum(p4) FROM fd_t WHERE q <= 99"

# ---- premises ---------------------------------------------------------------
check_text "premise: the selective query takes the fold (ANALYZE, post-#602)" \
"$(explain_line 'Columnar Batch Fold' "$QSEL" | grep -oE 'yes|no')" "yes"
# NGROUPS, never GROUPS: bash's builtin GROUPS (the caller's group-id array)
# silently IGNORES assignments, so a suite variable of that name reads as the
# runner's primary gid forever -- which burned two hours and a wrongly filed
# issue (#616) before cat -A, stream separation, and a single-line rewrite all
# failed to explain a value no assignment could change.
NGROUPS="$(q "SELECT count(*) FROM pgcolumnar.stats('fd_t'::regclass)")"
SURV="$(q "SELECT count(*) FROM fd_t WHERE q <= 1")"
CAND="$(q "SELECT count(*) FROM fd_t")"
check "premise: the selective predicate is genuinely selective (<= 2%)" \
"$([ $((SURV * 50)) -le "$CAND" ] && echo yes)" "yes"

# ---- correctness: three-way agreement --------------------------------------
FOLD_SEL="$(sq "$QSEL;")"
ROW_SEL="$(env PATH="$PGC_BINDIR:$PATH" psql -h 127.0.0.1 -p "$PGC_PORT" -U postgres -d "$PGC_DB" -At \
-c "SET pgcolumnar.enable_ungrouped_vector_agg = off" -c "$QSEL" 2>/dev/null | tail -1)"
HEAP_SEL="$(q "SELECT sum(p1), sum(p2), sum(p3), sum(p4) FROM fd_heap WHERE q <= 1")"
check "deferred fold == row path (selective)" "$FOLD_SEL" "$ROW_SEL"
check "deferred fold == heap mirror (selective)" "$FOLD_SEL" "$HEAP_SEL"
FOLD_HI="$(sq "$QHI;")"
HEAP_HI="$(q "SELECT sum(p1), sum(p2), sum(p3), sum(p4) FROM fd_heap WHERE q <= 99")"
check "fold == heap mirror (high survival)" "$FOLD_HI" "$HEAP_HI"

# ---- the work-done identity -------------------------------------------------
LOADS_LINE="$(explain_line 'Columnar Fold Payload Loads' "$QSEL")"
DEFER_LINE="$(explain_line 'Columnar Fold Deferred Groups' "$QSEL")"
check "the loads counter exists under ANALYZE" \
"$([ -n "$LOADS_LINE" ] && echo yes)" "yes"
LOADS="$(grep -oE '[0-9]+$' <<<"$LOADS_LINE")"
check "premise: the predicate has survivors (the identity is not 0 == 0)" \
"$([ "${SURV:-0}" -gt 0 ] && echo yes)" "yes"
check_num "deferred: loads == survivors x payload width (the identity)" \
"${LOADS:-0}" "$((SURV * NP))"
# cross-instrument premise: the EXPLAIN's own group total must agree with the
# stats() catalog count, and both must be plural -- whichever side misreports,
# this names it with the numbers in hand.
DEFER_Y="$(grep -oE 'of [0-9]+' <<<"$DEFER_LINE" | grep -oE '[0-9]+')"
check_num "premise: EXPLAIN group total == stats() group count" \
"${DEFER_Y:-0}" "${NGROUPS:-0}"
check "premise: the fixture spans several groups" \
"$([ "${DEFER_Y:-0}" -ge 3 ] 2>/dev/null && echo yes)" "yes"
check "deferred: every group deferred at 1% survival" \
"$(grep -oE '[0-9]+ of [0-9]+' <<<"$DEFER_LINE")" "$DEFER_Y of $DEFER_Y"

# ---- the adaptive gate declines high survival -------------------------------
DEFER_HI_N="$(explain_line 'Columnar Fold Deferred Groups' "$QHI" | grep -oE '[0-9]+ of' | grep -oE '[0-9]+')"
check_num "adaptive: only the optimistic first group deferred at 99% survival" \
"${DEFER_HI_N:-0}" "1"
LOADS_HI="$(grep -oE '[0-9]+$' <<<"$(explain_line 'Columnar Fold Payload Loads' "$QHI")")"
SURV_HI="$(q "SELECT count(*) FROM fd_t WHERE q <= 99")"
check "adaptive: eager groups load more than survivors alone (work measured, not assumed)" \
"$([ "${LOADS_HI:-0}" -gt $((SURV_HI * NP * 99 / 100)) ] && echo yes)" "yes"

pgc_summary
1 change: 1 addition & 0 deletions test/run_all_versions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ SUITES=(
native_fetch_interrupt
native_fetch_position
native_fetch_projection
native_fold_deferral
native_fold_skipguard
native_format
native_gap
Expand Down
Loading