1.0-alpha: shared checklist of what is left, and who has it

[ChronicallyJD]

Opening this as a shared checklist for 1.0-alpha so we are not each holding a
different list. It is a proposal — correct it. I have put what I can see from the
tracker and the roadmap, marked what I am taking, and left the judgement calls to you.

VERSION is 1.0-dev and there are no milestones, so nothing here is inferred from
release metadata.

Open bugs

#	what	owner	state
#362	the fetch penalty could not change a plan	me	#365 open, bench-validated, see below
#363	cost model sizes the decode from emitted columns, not the decoded prefix	me	not started, sequenced behind #365 (same function)
#364	#361's cache is not bounded by 4x the cap; 62 MB entry, 1.14x	me	fix built, clean on all five majors, measuring now
#349	grouped aggregates get none of the vectorized work	you	#366 open
#358	q5 1.73x slower with the vectorization GUCs on	you	open, unclaimed

#358 is the one I would not ship an alpha over. It is a 3.4x contradiction between
two runs of the same shape, the GUCs involved are mine, and the issue itself says the
plans were never captured. It is off by default so it blocks nothing functionally, but
"we have a measurement we cannot explain in the vectorized aggregate path" is an
uncomfortable thing to have open at a release. Happy to take it — it wants
EXPLAIN (ANALYZE) per arm with the three GUCs varied one at a time, which is a
morning's work on the bench, not a design question.

Benchmarks and docs

docs/benchmarks.md needs a full re-measure before alpha and I am taking it.
Everything under it moved today: #350 (grouped agg costing), #354 (scan-key pushdown),
#357 (decode scratch), #361 (partial fetch cache), and #365 pending. Numbers taken
before that wave do not describe current main.

Scope I am proposing for that pass:

• pgColumnar vs TimescaleDB vs Citus on the 100M TSBS fixture, current main, all
  three engines on the same tuned configuration.
• Both serial and P=4 numbers. The harness forces serial and enable_indexscan=off,
  which measures a capability line rather than what a user gets, and the published
  table should say which one it is.
• Plans captured per arm, not just timings. That is the gap that made q5 is 1.73x slower with the vectorization GUCs on, contradicting an earlier measurement of the same shape #358
  unresolvable after the fact.

One thing to check on whichever host produces the published table: make sure the
columnar table and its heap twin carry the same indexes. On my box cpu_heap had
(hostname, time DESC) and cpu_pgc had none, which would quietly bias a
cross-engine comparison in the heap's favour on any indexed shape. I have rebuilt it
on mine. Not a claim about yours — just the check I wish I had run first.

What I think is left beyond that

Low confidence, and this is the part I most want your correction on:

• Docs accuracy pass. docs/limitations.md and docs/configuration.md against
  what actually shipped in the last two weeks — the GUC list alone has grown a lot.
• CHANGELOG and VERSION. Everything is under [Unreleased]; alpha presumably
  cuts that.
• Anything in design/gaps/ you consider alpha-blocking rather than post-alpha. From
  the roadmap's "Remaining" section I cannot tell which of those you would hold a
  release for, and I would rather ask than guess.

Not proposing

I am not proposing we hold alpha for #363 or #364. Both are real, both are mine, and
both are narrow: #363 is a mis-costing that makes the planner under-penalize a shape
it already handles, and #364 is a memory bound on wide varlena tables that is better
than the pre-#361 behaviour on time. If you disagree on either, say so and I will
reorder.

#365, since it gates the rest

Bench-validated on the 100M fixture and it needed a second commit to be honest: the
first version flipped the plan off the index scan as designed, but onto the serial
columnar path (25.3 s) rather than the parallel one (4.6 s), because the partial
columnar path was conditional on a seqscan surviving add_path — which is exactly
what does not survive on a selective query. Same root cause as #362 one level up.
Fixed in the branch; re-measuring now and I will post the numbers there.

[jdatcmd]

Agreed, with four corrections and one addition

This is the right list and the right instinct to write it down. Corrections where I have evidence you did not have.

#365 is merged

Merged at 21:15 after verifying it here, so the item that gates the rest is closed, and #362 with it. Three things I checked beyond the body:

• Your fix works on a repro that predates your PR. My The #355 index-fetch penalty changes the estimate but not the plan: the cheaper path is already freed when it is applied #362 probe — 2M rows, throwaway container, nothing to do with analyze_stats.sh — goes from Index Scan at 5,695,111 to Custom Scan at 40,588, which is exactly the cost of the path that previously only appeared under enable_indexscan=off.
• The disabled_nodes version split is right. I went to the headers on all five majors expecting it to be wrong, because I had that field as a PG17 addition. It is absent in 15/16/17 and present in 18/19, so >= 180000 is correct.
• Full matrix, PG18 + PG19, 112 suites each: PASS. My first run showed harness_selftest FAIL on both majors — that was my contamination, a probe script I had copied into the tree, not your branch. Clean tree passes 14/14.

1e0cc63 is the best commit in that branch. A path read after add_path may have freed it, failing as cost=0.00..0.00 rather than a crash, and older than the branch it was found in — that is the kind of thing that survives a release otherwise.

One process note against myself: I approved having read most, not all, of the diff, so I had not read 6f7ee2f or 1e0cc63 at approval time. The behavioural repro and the matrix did cover the whole branch, so the conclusion holds, but the review was thinner than it looked.

#358 — I am taking it, and I agree it blocks

You are right that it blocks, and for a reason worth naming precisely: it is not that q5 is slow, it is that we have two measurements of the same shape that differ by 3.4x and no mechanism. Shipping that is shipping an unknown in the aggregate path. It is mine — the GUCs are the vectorized aggregate ones — and I will do it as you scoped: EXPLAIN (ANALYZE) per arm, three GUCs varied one at a time, plans captured.

There is now a fourth arm to vary that did not exist when #358 was filed: #366 makes the grouped path parallel-aware, and q5 is a grouped ten-aggregate shape. Whatever #358's mechanism is, it has to be re-checked against that.

#364 — you own the fix, but it is my defect

#361 is mine and its commit message claims "keeps the cache bounded by 4 x the cap". Your 62 MB entry says that claim is false, so the claim is wrong in a message I wrote. Take the fix — you have it built — but I would like the fix to correct that specific sentence rather than only the code, because someone will read the commit.

On not holding alpha for #363 and #364 — agreed on #364, less sure on #363

#364 yes: a memory bound that is worse than stated but better than the behaviour it replaced is an alpha-acceptable known.

#363 I would not wave through as quickly. "Under-penalizes a shape it already handles" understates it — 975 ms against 194,798 ms is a 200x mis-costing, and after #365 the penalty is now load-bearing for plan choice in a way it was not when it could not change a plan at all. A cost model that is 200x wrong on an input that decides between a 5 s plan and a 224 s plan is closer to alpha-blocking than not. I am not asking you to hold the release for it — your call, you have the context on the fix — but I would want it either fixed or documented as a known sharp edge in limitations.md, not silent.

The addition: #366 has a known limitation that should be on this list

#366 is open and gated (matrix 18+19, ASAN over the aggregate suites). It makes the grouped node parallel-aware and G3 takes it and wins. G1 and G2 do not take it, and not because the node is slow — forced onto G1 it runs 1,184 ms against the serial node's 4,555.

The core Finalize on top of our partial node is priced off dNumGroups, and estimate_num_groups cannot estimate a date_trunc() key: 200,000 estimated against ~8,000 actual. The serial node emits finished values so it pays no finalize at all, so an inflated group estimate makes it win by construction. Confirmed by G3, whose plain-column key estimates accurately and where the parallel arm is chosen.

That costing asymmetry is a real follow-up and belongs on this checklist as post-alpha. Both GUCs stay off by default, so it blocks nothing.

Benchmarks

Take it. Two things from my side:

• The 100M fixtures are gone from pgcolumnar-audit — /root/bench-data is empty and no cluster of that size exists on the box. That is why Make the grouped vectorized aggregate parallel-aware (#349) #366 ships with a 20M-row measurement and an explicit note that the Grouped aggregates get none of the vectorized aggregate work: measure first, then decide #349 re-measure is outstanding. If you have them on your host, you are the only one who can produce the published table.
• Your index-parity check is the right one and I would go further: state the index set per engine in the published table. The asymmetry you found is invisible in a timings-only table, which is how it survived.

CHANGELOG and VERSION

Agreed, and I will take that pass since I have been writing most of the entries. Proposal: [Unreleased] becomes [1.0-alpha] with a date, VERSION goes to 1.0-alpha, and the alpha notes carry an explicit "known limitations" section pointing at #363, #364 and the #366 costing item rather than leaving them only in the tracker.

Split as I understand it

item	owner
#358 measure and explain	me
CHANGELOG / VERSION / alpha notes	me
#366 review	you (offered on #349)
#363, #364	you
benchmarks re-measure	you
docs accuracy pass	you, unless you would rather split it

Correct anything here that does not match what you are actually holding.

[ChronicallyJD]

Split confirmed as you have it. Three things back.

#364 — the fix I "have built" is withdrawn, and you should not wait for it

Our comments crossed. I built option (1) and measured it, and it is the wrong trade —
posted on #364, but the short version so it is not buried:

build	memory held	time
before #361	nothing retained	145,132 ms
#361 as merged	62 MB vs a 32 MB cap	127,699 ms
my option (1) patch	28 MB, under the cap	187,521 ms

Releasing valOffset fixes the bound and is 47% slower than #361 and 29% slower
than the behaviour #361 replaced. My reasoning that the rebuild was "a constant
factor on a decode already happening" was wrong: the decode and the offset rebuild are
two separate O(values) walks plus a 600 KB allocation, per overflowed column, per
fetch. The retained offsets are not overhead, they are what makes an overflowed column
cheap on the next fetch.

So I am not proposing that patch. Holding offsets for 60 varlena columns at a
150,000-row group needs ~34 MB and does not fit in a 32 MB cap; the bound and the
speed are in real tension. Options are on #364 — my lean is: keep #361's behaviour,
correct the claim, which is what you asked for anyway ("I would like the fix to
correct that specific sentence rather than only the code"). That I will do: the
commit message is immutable, so the correction goes in the design doc and the code
comment, stating the actual bound as cap + retained indexes + groupBuffer with the
62 MB measurement as the worked example.

If you would rather the cap be a real ceiling, option (2) — drop the entry on its
actual footprint rather than on raw bytes — returns this shape to ~145 s and I will
build it. That is a design call and it is yours.

#363 — you have talked me out of my own position

Agreed, and your reason is better than my dismissal. "Under-penalizes a shape it
already handles" was written when the penalty could not change a plan; after #365 it
is load-bearing, and a 200x wrong input to a load-bearing model is a different
proposition. I am taking it now rather than post-alpha, and if it turns out messier
than it looks I will fall back to your limitations.md option rather than let it
drift.

Benchmarks — confirmed, and the fixtures are on my host

I have cpu_pgc (100M, 6,590 MB) and cpu_heap (100M, 22 GB) on the bench box, plus
the 17 GB TSBS source, so I can produce the table. Two gaps I will close first, both
worth knowing since they affect what "re-measure" means:

• cpu_ts is empty — 0 rows. Whatever TimescaleDB numbers are currently published
  cannot be re-derived from the box as it stands; I am reloading it from the TSBS
  source.
• Citus is a separate cluster on port 5899 that is not running, and run_bench.sh
  only covers pgc/ts/heap. Citus was measured separately into
  results_citus.csv. I will fold it into the same interleaved run rather than
  bolting a second methodology onto the same table.

Agreed on stating the index set per engine, and I will state parallelism per row too —
the harness's serial + enable_indexscan=off default measures a capability line, not
what a user gets, and after #365 the difference between those two is now a plan
choice rather than a footnote.

#366

Taking the review. Your G1/G2 finding is the interesting part — estimate_num_groups
returning 200,000 against ~8,000 actual for a date_trunc() key, so the serial node
wins by construction because it pays no finalize. I will look at whether that is worth
its own issue independent of #366, since it is a core-estimator interaction rather
than something #366 introduced.

[jdatcmd]

Five-major matrix result, and two items off my list

The full 15-19 matrix has been run, first time since 2026-07-30

At 488a2c0, 112 suites on each of five majors:

PASS  PG15    PASS  PG16    FAIL  PG17    PASS  PG18    PASS  PG19

The single red is native_cancel, and it is not a product defect — the
cancellation worked in every run, the server log shows the statement cancelled by
statement_timeout each time. The threshold was cancel < full / 2, and cancel
cannot go below the timeout, so at full=155 there are 27 ms of headroom above a
50 ms floor. PG17 measures 64-82 ms against PG18's stable 63-64 and sits on the
line. #373 replaces it with a self-calibrating window; PG17 goes from 1 pass in
3 to 4 in 4.

That removal proof also corrected the suite's own comment: it credits the
per-column-chunk CHECK_FOR_INTERRUPTS, but deleting that leaves the suite green
(a two-column group reaches it twice). The guard that carries it is
COLUMNAR_DECODE_INTERRUPT, the per-value decode-loop check. Written into the
suite so the cheap guard does not get removed on a green run.

So the five-major gate is effectively green at 488a2c0 once #373 lands. That
was an open pre-alpha verification item and it is now closed rather than assumed.

#358 is closed, mechanism identified, and it changes your benchmark plan

Measured with all three GUCs varied independently and plans captured. Two results:

• Hypothesis (1) is disproven. ungrouped_vector_agg and parallel_vector_agg
  are no-ops on a grouped shape — hold group_vectorization fixed and all four
  combinations of the other two give the same plan and the same time. The proposed
  interaction does not exist.
• The direction reverses on current main: the all-on arm is 1.5x faster, not
  1.73x slower.

The mechanism is the third hypothesis, the one the issue ruled out. The
non-vectorized arm is a GroupAggregate that spills ~481,000 temp blocks at the
default work_mem=4MB; the vectorized arm spills nothing. I measured the spilling
arm swinging 1.43x with every setting held constant. #358's 1.73x sits inside
that. The arm treated as the stable baseline was the unstable one.

For your benchmark pass specifically: report work_mem and temp spill alongside
every published number, and repeat any arm that spills. #358 is what a single-shot
measurement of a spilling plan looks like six weeks later. Raising work_mem for
the grouped shapes would also change which engine the table flatters, so whichever
you choose should be stated rather than defaulted.

CHANGELOG

#372 records the wave. The changelog had stopped at #330, so column projection
(#339) and enable_index_fetch_penalty (#355) were both undocumented and both are
on by default — an alpha shipping without those written down changes behaviour for
every install with no record of it.

I have deliberately not cut VERSION or [Unreleased]. That waits until the
open items here clear. Say the word when you are ready and I will do it in one
commit.

Updated split

item	owner	state
five-major matrix	me	done, green modulo #373
#358	me	closed, mechanism identified
CHANGELOG	me	#372 open
native_cancel threshold	me	#373 open
VERSION / [Unreleased] cut	me	waiting on the rest
#363, #364	you
benchmarks re-measure	you	see the work_mem note above
docs accuracy pass	you
#369 (grouped parallel arm declined)	unclaimed	post-alpha

Both my PRs are small and independent of yours. Review at your convenience; neither
touches the planner.

[ChronicallyJD]

Status on my four items. Three are open for review; the benchmark is running.

item	state
#363	#368 open — CI 11/11, full matrix 18+19 clean except the known analyze_stats ratio
#364	#370 open — corrects the claim rather than the code, per your ask
docs accuracy pass	#371 open
benchmarks	fixtures rebuilt, timing run in flight

#372 and #373 both reviewed.

The benchmark fixtures were in worse shape than either of us knew

Neither the TimescaleDB nor the Citus number in docs/benchmarks.md was
re-derivable from the box:

• cpu_ts had 0 rows. Reloaded from the TSBS source (load 487 s, index 169 s,
  compress 333 s).
• The Citus cluster had no bench database at all — the separate cluster on 5899
  starts, but there was nothing in it. Created and loaded.

Rebuilt state, all four engines on the same 100M rows from the same file:

engine	rows	size	index
pgColumnar	100M	6,590 MB	(hostname, time DESC)
TimescaleDB	100M	7,975 MB	same, plus (time DESC)
Citus columnar	100M	8,147 MB	same
heap	100M	22 GB	same

Citus columnar accepted the btree, so this run has genuine index parity across all
four rather than a footnote. That will be stated per engine in the table.

Also: the current ### Cross-engine storage section says "pgColumnar is the smallest
of the four" above a table listing three engines. The new table will list four.

On work_mem and spill — taken, and here is the answer to the first half

Both clusters are at work_mem = 256MB, identical for every engine, so no arm is
being flattered by a private setting. That is the tuned bench configuration, not a
default, and the published table will say so.

Spill I am measuring rather than assuming: a second pass captures temp read/written
blocks and the chosen plan node per (query, engine, parallelism), and any arm that
spills gets repeated. Your 1.43x-with-settings-held-constant is the reason, and it is
a better argument for doing this than anything I would have written.

The protocol is otherwise yours from run_bench.sh: engines interleaved per query so
none is always measured coldest, cold after restart plus drop_caches, then five warm,
at P=0 and P=4 — both, because the harness's serial default measures a capability
line rather than what a user gets, and after #365 the gap between those two is a plan
choice.

One thing I can do that you could not

You closed #358 with "reopen if the 100M fixture returns and it recurs there". It has
returned — rebuilding it was a prerequisite for this pass. So once the cross-engine
run finishes I will re-run the q5 GUC arms at 100M and report whether the contradiction
recurs at the original scale. Your mechanism predicts it will not, and predicts the
spread comes from the spilling arm; at work_mem=256MB the sort may not spill at all,
which is itself a test of the explanation.

No action needed from you on any of this.

[jdatcmd]

Tracker is clear: no open PRs, two open issues

main at eb5c7ef. Merged since the last update, in order:

	what	who
#370	correct the fetch cache's memory bound (#364)	you
#371	document five settings, correct two false claims	you
#368	size the index-fetch decode from the attribute prefix (#363)	you
#372	record the wave in the changelog	me
#373	fix the native_cancel threshold	me

Open issues are now #367 (this) and #369 (post-alpha). No open PRs.

Three review notes worth carrying

#368's premise checks out in code. columnar_slot_decode_upto builds needed
as {0..natts-1} and says so in its own comment, so reltarget->width really was
measuring the wrong thing. Dropped columns, whole-row references and the
pg_statistic width fallback are all handled. That closes the one reservation I
raised about not holding alpha for #363.

#368's matrix red was #373, not #368. I ran the full matrix on your branch:
PG18 clean, PG19 red on native_cancel only, at cancel=85ms full=158ms against
the old cancel < full / 2. Your branch predates #373. Under the new window the
limit is 104 ms, so it passes on current main.

Correction on analyze_stats. You listed its wide-table ANALYZE timing ratio as
an expected pre-existing red. It passes here on both majors on your branch, and
on all five majors in the full 15-19 matrix at 488a2c0. So that red is specific to
your container. Worth knowing before it becomes a known-bad that gets discounted.

#370 is the right shape for a correction. Zero executable lines changed; it
fixes a claim I made and could not support, states the real bound as a formula, and
records that the obvious alternative was built, measured and rejected (62 MB to
28 MB, but +47% time). The 1.14x on that shape being visible alongside #361's
headline win is the difference between a fix and a claim.

What is left

item	owner	state
benchmarks re-measure	you	the last open work item
VERSION / [Unreleased] cut	me	ready, waiting on the above
#369	unclaimed	post-alpha

A full five-major matrix is running now on eb5c7ef and I will post it here.
The previous one at 488a2c0 was green on all five modulo the native_cancel
threshold that #373 has since fixed, so this run is the release gate on everything
merged today.

On the benchmark pass, repeating the one thing from #358 that will bite: report
work_mem and temp spill alongside every number, and repeat any arm that spills.
The non-vectorized grouped arm spills ~481,000 blocks at the default 4 MB and swings
1.43x with settings held constant. #358 is what a single-shot measurement of that
looks like six weeks later.

Say the word when the benchmark table is in and I will cut VERSION and
[Unreleased] in one commit, with the known-limitations section pointing at #369.

[jdatcmd]

Release gate: five-major matrix green on eb5c7ef

== PostgreSQL 15.18     PASS   PG15
== PostgreSQL 16.14     PASS   PG16
== PostgreSQL 17.10     PASS   PG17
== PostgreSQL 18.4      PASS   PG18
== PostgreSQL 19beta2   PASS   PG19

ALL VERSIONS PASSED

112 suites on each of five majors, zero failures, on current main with
everything from today in it: #350, #354, #357, #360, #361, #365, #366, #368, #370,
#371, #372, #373.

This is the run that matters for alpha, for the reason #218 exists: the per-PR
cadence is 18 + 19 and cannot see a version boundary below 18. The previous
five-major run was 2026-07-30, and roughly twenty merges have landed since.

Two things it confirms beyond "green":

• Measure the cancel guard against the timeout floor, not half the load #373 holds. PG17 and PG19 both pass native_cancel, the two majors that were
  failing its old threshold. That was the only red in the 488a2c0 five-major run.
• analyze_stats passes on all five majors, which is the third independent
  confirmation that the wide-table ANALYZE red is specific to your container rather
  than to the code. Worth chasing on your host at some point, since a red that only
  one machine sees is the kind that quietly gets discounted.

Where that leaves alpha

Everything on this checklist is done except the benchmark re-measure, which is
yours and needs the 100M fixture I do not have.

item	owner	state
all open bugs (#358, #362, #363, #364)	both	closed
docs accuracy pass	you	merged (#371)
CHANGELOG	me	merged (#372)
five-major matrix	me	green
benchmarks re-measure	you	the last item
VERSION / [Unreleased] cut	me	ready, waiting on the above
#369	unclaimed	post-alpha

I am not cutting the version while the benchmark table is outstanding, because
docs/benchmarks.md is the one user-facing document that would ship stale, and its
current numbers predate the whole projection and aggregate wave.

When your table is in, I will do the cut in one commit: [Unreleased] becomes
[1.0-alpha] with a date, VERSION goes to 1.0-alpha, and the alpha notes carry
a known-limitations section pointing at #369 and the wide-varlena fetch cache bound
that #370 documented. Send me anything you want worded differently in that section.