refactor: columnar_metadata gets its own header, the first of six (#496) - #546
Conversation
…mmandprompt#496) columnar.h declares 137 functions. 84 of them have exactly one consumer outside their defining file -- a private arrangement between two files that the other twenty are forced to recompile for, and that anyone reading the header has to scan past to reach the 49 that are genuinely shared. This moves the largest single group: the 22 declarations defined in columnar_metadata.c, into src/columnar_metadata.h beside it. The shared vocabulary they take -- the Native*Metadata structs, the GUCs, the format constants -- stays in columnar.h, which the new header includes. The census in commandprompt#496 was stale and understated the problem. Re-measured on 3e86a7e: when filed today columnar.h lines 957 1045 extern declarations 159 164 single-consumer 79 84 so the header gained interface faster than commandprompt#527 removed it. Re-measuring also caught an error in my own tool. Its first version attributed PgColumnarBloomBuild and PgColumnarEncodeChunk to columnar_write_state.c, which does not define either: it took whichever file matched first in directory order. Fixed to require a definition at column 0, which is the style this tree uses, and the ranking changed -- columnar_metadata went from outside the top three to the largest module at 22, and columnar_write_state from 25 to 11. The wrong module would have been extracted first. Gate: builds on pg15a, pg16a, pg17a, pg18a and pg19a, make clean between each (commandprompt#536), zero warnings and zero errors on all five. That is the gate that matters for a declaration-only change: nothing here alters a line of executable code, and a consumer left without the declaration it needs fails to compile rather than misbehaving. native_vecdecode 24/24 and harness_selftest 54/54 on pg18a besides. Five modules remain -- reader (12), write_state (11), storage (10), customscan (6), delete_vector (6). One PR each: 84 declarations in a single diff is not a reviewable change.
ChronicallyJD
left a comment
There was a problem hiding this comment.
Reviewed by checking the claims rather than reading the diff. The split is
clean and I have no objection to its content. One structural finding, and it is
about #546 and #547 together rather than either alone.
Verified
Every new include is needed. For each .c that gained
#include "columnar_metadata.h", counted how many of the header's 22 symbols it
actually references:
columnar_metadata.c 22 columnar_vacuum.c 7
columnar_write_state.c 6 columnar_reader.c 3
columnar_projection.c 2 columnar_tableam.c 2
columnar_vector.c 1 columnar_visibilitymap.c 1
No include added that nothing uses, which is the usual way a mechanical split
adds churn.
The stated rationale holds, for all 22. The header says every declaration had
exactly one consumer outside its defining file. Counted per symbol across
src/*.c excluding columnar_metadata.c: not one of the 22 has more than one
consumer. That is a checkable claim and it checks out, which is worth saying
because it is the entire justification for the split.
The finding: #546 and #547 are not stacked, and they collide
Both are titled as a chain — "the first of six", "the second of six" — but both
are single commits based on main, and neither contains the other.
Trial-merged them onto main in sequence:
merge 546 -> clean
merge 547 -> CONFLICT in src/columnar_reader.c
src/columnar_tableam.c
src/columnar_vector.c
Not columnar.h, as I first assumed — the removed hunks there are far enough
apart to auto-merge. It is the #include lines, added at the same place in the
consumers both PRs touch.
With six planned this compounds: each pair that shares a consumer collides the
same way, and the conflicts land in .c files rather than in the header being
split, so they read as unrelated to the change.
Either stack them (547 based on 546) or expect to rebase each one after the
previous merges. I have no view on which — I hit the auto-close trap stacking
#534 this morning and would not push anyone toward stacking without saying so.
Not a finding
A pure header split has no behaviour to assert, so "it builds and the suites
pass" is the right standard and CI covers it. I did not ask for a test.
…mandprompt#496) The 12 declarations defined in columnar_reader.c with exactly one consumer outside it: the fold entry points, the liveness cache, the projection and filtered-read calls. Consumed only by customscan, tableam and vector. Gate: builds on pg15a, pg16a, pg17a, pg18a and pg19a, make clean between each (commandprompt#536), zero warnings and zero errors on all five. native_vecdecode 24/24, native_fold_skipguard 4/4 and harness_selftest 54/54 on pg18a. NOT independent of commandprompt#546, and I claimed it was. I measured the closest pair of declaration removals in columnar.h between the two patches -- 49 lines, well outside git's 3-line context -- and concluded they could both sit on main. The gate applied both and refused: error: patch failed: src/columnar_reader.c:14 error: patch failed: src/columnar_tableam.c:13 error: patch failed: src/columnar_vector.c:36 The collision is not in the header at all. Both patches add their #include immediately after #include "columnar.h" in the three files that consume BOTH modules -- reader, tableam and vector -- so they edit the same anchor line. My arithmetic was correct about the thing it measured and blind to the surface that actually collides. This still bases on main rather than stacking on commandprompt#546, deliberately. Stacking is what auto-closed commandprompt#534 this morning when its base branch was deleted on merge, and the conflict here is three adjacent include lines: whichever of the two lands second wants a rebase that is mechanical and takes a minute. That is the cheaper of the two failure modes.
) write_state (11), storage (10), customscan (6) and delete_vector (6) — the 33 remaining declarations with exactly one consumer outside their defining file. ONE PR rather than four, and the reason is how commandprompt#546 and commandprompt#547 were reviewed: by running checks — every new include used, no moved symbol having more than one consumer — not by reading declarations. Correctness here is established by script, so four diffs are no easier to review than one, and one diff collides with commandprompt#546 and commandprompt#547 once instead of up to fifteen times. Every pair of module PRs sharing a consumer conflicts on the #include line, which is what made a six-PR series expensive. Not stacked, deliberately: a stacked PR auto-closes when its base is deleted on merge (commandprompt#534, this morning). Merge in any order; whichever is second wants a one-minute rebase of the include lines. A defect in my splitting tool, found by the count assert and worth recording because it silently produced a WRONG module assignment rather than an error: columnar.h:353 is extern void PgColumnarEnsureStorageRow(Relation rel);\t/* pre-create ... */ which ends in "*/", not ";". The block scanner ran to the next line ending in ";" and swallowed the declaration below it, filing PgColumnarReserveRowNumbers — a storage symbol — under write_state.h. It compiles either way, because a declaration is visible from whichever header the consumer includes; only the "matched 9 of 10" assert on the NEXT module caught it. Terminator detection now strips trailing comments. I checked whether it reached the open PRs. It did not: commandprompt#546 and commandprompt#547 carry exactly their intended sets, 22 and 12, nothing extra, nothing missing, nothing left behind in columnar.h. That is luck — neither module's declarations happened to sit beside one with a trailing comment. All four verified exact: 11/11, 10/10, 6/6, 6/6, no extra symbol, none missing, none stranded. columnar.h goes 1045 -> 997 lines and 164 -> 131 externs. Gate: pg15a, pg16a, pg17a, pg18a, pg19a, make clean between each (commandprompt#536), zero warnings and zero errors on all five. native_vecdecode 24/24, harness_selftest 54/54, replication 41/41 on pg18a.
) write_state (11), storage (10), customscan (6) and delete_vector (6) — the 33 remaining declarations with exactly one consumer outside their defining file. ONE PR rather than four, and the reason is how commandprompt#546 and commandprompt#547 were reviewed: by running checks — every new include used, no moved symbol having more than one consumer — not by reading declarations. Correctness here is established by script, so four diffs are no easier to review than one, and one diff collides with commandprompt#546 and commandprompt#547 once instead of up to fifteen times. Every pair of module PRs sharing a consumer conflicts on the #include line, which is what made a six-PR series expensive. Not stacked, deliberately: a stacked PR auto-closes when its base is deleted on merge (commandprompt#534, this morning). Merge in any order; whichever is second wants a one-minute rebase of the include lines. A defect in my splitting tool, found by the count assert and worth recording because it silently produced a WRONG module assignment rather than an error: columnar.h:353 is extern void PgColumnarEnsureStorageRow(Relation rel);\t/* pre-create ... */ which ends in "*/", not ";". The block scanner ran to the next line ending in ";" and swallowed the declaration below it, filing PgColumnarReserveRowNumbers — a storage symbol — under write_state.h. It compiles either way, because a declaration is visible from whichever header the consumer includes; only the "matched 9 of 10" assert on the NEXT module caught it. Terminator detection now strips trailing comments. I checked whether it reached the open PRs. It did not: commandprompt#546 and commandprompt#547 carry exactly their intended sets, 22 and 12, nothing extra, nothing missing, nothing left behind in columnar.h. That is luck — neither module's declarations happened to sit beside one with a trailing comment. All four verified exact: 11/11, 10/10, 6/6, 6/6, no extra symbol, none missing, none stranded. columnar.h goes 1045 -> 997 lines and 164 -> 131 externs. Gate: pg15a, pg16a, pg17a, pg18a, pg19a, make clean between each (commandprompt#536), zero warnings and zero errors on all five. native_vecdecode 24/24, harness_selftest 54/54, replication 41/41 on pg18a.
First of six, for #496.
columnar_metadata.c's 22 single-consumer declarations move tosrc/columnar_metadata.hbeside it.The census, re-measured, because the issue's numbers were stale
3e86a7e)columnar.hlinesexterndeclarationsThe header gained interface faster than #527 removed it. Of 137 function declarations, 84 have exactly one consumer outside the defining file and 49 are genuinely shared.
What moved, and what did not
The 22 functions defined in
columnar_metadata.cand used by exactly one other file. The shared vocabulary they take — theNative*Metadatastructs, the GUCs, the format constants — stays incolumnar.h, which the new header includes. Nothing is madestatic, so #527's disqualification does not apply here: this relocates declarations, it does not narrow linkage.Verified mechanically: each of the 22 appears exactly once in the new header and zero times in
columnar.h.A defect in my own measurement, since it decided which module went first
The first version of my census attributed
PgColumnarBloomBuildandPgColumnarEncodeChunktocolumnar_write_state.c, which defines neither. It picked whichever file matched first in directory order. Requiring a definition at column 0 — the style this tree uses, return type on its own line — changed the ranking:columnar_metadata.ccolumnar_reader.ccolumnar_write_state.cThe wrong module would have been extracted first. It was caught only because
PgColumnarBloomBuildliving incolumnar_write_state.cis absurd on sight; two plausible filenames and I would have believed it.Gate
make cleanbetween each, per #536. Plusnative_vecdecode24/24 andharness_selftest54/54 on pg18a.The five-major build is the gate that matters here and I want to be explicit about why rather than pad the list: nothing in this diff alters a line of executable code. The failure mode of a declaration move is a consumer left without a declaration it needs, and that fails to compile — loudly, on whichever major first sees it — rather than misbehaving at run time. Zero occurrences of
error:orwarning:across five full builds is the direct evidence for that, and a suite run is corroboration rather than the proof.Remaining
reader(12),write_state(11),storage(10),customscan(6),delete_vector(6) — one PR each. 84 declarations in a single diff is not a reviewable change, and each module's consumers are a different set of files.I have not merged this and will not.