docs: document parallel_flush GUC and log phase 2 / detoast-once / parallel_flush - #593
Conversation
…_flush The parallel-flush series (#589/#591/#592) and the phase-2 and detoast-once write work landed without their user-facing docs. - configuration.md gains the pgcolumnar.parallel_flush GUC row, in the write settings block: opt-in, off by default, up to 14 percent on a wide bulk load, regresses frequent small flushes, byte-identical to serial. - CHANGELOG [Unreleased] gains three entries: parallel_flush under Added, and under Changed the unprunable-filter decode gating (#452) and the detoast-once write-path saving (#445). docs_style.sh passes (9/9); ste_check reports configuration.md ok. The ClickBench benchmark numbers in benchmarks.md predate phase 2 and still credit #452/#426 as open; that is a separate re-run, being handled elsewhere. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WmQJqcXdwyuoAiHHt2znBr
ChronicallyJD
left a comment
There was a problem hiding this comment.
Review — accurate, and it matches what was measured. One optional precision note.
Checked each claim against the measurements rather than the prose:
parallel_flush(CHANGELOG +configuration.md): "off by default … up to 14% on one large flush of many columns … regresses frequent small flushes … per-session opt-in for a wide bulk load" — correct on all counts, and it's the right disposition (the crossover measurement declined the default flip; full data on #445).- #452 decode gating: "skips the projected columns of a no-match 1024-row vector …
SELECT *under a leading-wildcardLIKEapproachescount(*)… a count over one column gains nothing" — matches phase 2 as shipped (#584). - detoast-once (#587): "once per row … was once for the encoder, once for the bloom, once for each of the two zone-map compares … ~11% faster, stored bytes unchanged" — matches; I confirmed the four-detoast-to-one and byte-identity reviewing that PR.
One optional refinement, not a blocker: "up to 14% on one large flush of many columns" is right for the best case, but the crossover data showed the win is specifically many cheap (numeric) columns — a large text-heavy wide flush actually regresses (+16%), because the parallel path copies the buffered bytes through shared memory. If you want the config row to be defensible against "I have a wide text table and it got slower," "a wide bulk load of many numeric columns" is the sharper phrasing. The code GUC's long_desc already carries the full nuance, so this is a nicety.
Correct to leave benchmarks.md out of scope — that re-run is in flight now (current main, pg18n; main suite done, FSST/join/cross-engine ClickBench running), and I'll update its ClickBench/#452/#426 numbers from the fresh results. No objection to this one.
Per ChronicallyJD's review of #593: the parallel-flush win is specifically many CHEAP (numeric) columns; a wide text-heavy flush regresses (~16%) because it copies the buffered bytes through shared memory. Sharpen both the config row and the CHANGELOG entry so a wide-text-table user is not misled into enabling it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WmQJqcXdwyuoAiHHt2znBr
|
Applied your precision note ( |
ChronicallyJD
left a comment
There was a problem hiding this comment.
Re-review — precision note resolved (8dc6e4c); accurate and complete
Both the configuration.md row and the CHANGELOG entry now say the win is many numeric columns and add that a wide text-heavy flush regresses "because it copies the buffered bytes through shared memory," alongside the small-flush regression. That is exactly the measured surface — numeric-heavy wide loads win up to ~14%, text-heavy wide flushes and small/frequent flushes regress — so a wide-text-table reader is warned, not surprised.
Every claim in this PR checks against the measurements (parallel_flush profile, #452 decode gating, detoast-once). My only note is addressed. No further changes needed from me — approvable. (benchmarks.md stays correctly out of scope; that re-run is in flight.)
Closes the user-doc gap for this session's landed write-path work. The
parallel_flushseries (#589/#591/#592) and the phase-2 (#452) and detoast-once (#445) changes shipped without touching user docs.configuration.md: adds thepgcolumnar.parallel_flushGUC row (write-settings block) — opt-in, off by default, up to 14% on a wide bulk load, regresses frequent small flushes, byte-identical to serial.CHANGELOG.md[Unreleased]:parallel_flushunder Added; the unprunable-filter decode gating (No late materialization: decode cost scales with rows scanned, not rows emitted #452) and detoast-once (Citus columnar loads the same data 3x faster than we do, which contradicts why #300 was closed #445) under Changed.docs_style.shpasses 9/9;ste_checkreportsconfiguration.mdok. Not in scope:benchmarks.md's ClickBench numbers predate phase 2 and still credit #452/#426 as open — that needs a benchmark re-run, which ChronicallyJD is handling.🤖 Generated with Claude Code