Skip to content

refactor: columnar_metadata gets its own header, the first of six (#496) - #546

Merged
ChronicallyJD merged 1 commit into
commandprompt:mainfrom
ChronicallyJD:refactor/496-metadata-header
Aug 10, 2026
Merged

refactor: columnar_metadata gets its own header, the first of six (#496)#546
ChronicallyJD merged 1 commit into
commandprompt:mainfrom
ChronicallyJD:refactor/496-metadata-header

Conversation

@ChronicallyJD

Copy link
Copy Markdown
Collaborator

First of six, for #496. columnar_metadata.c's 22 single-consumer declarations move to src/columnar_metadata.h beside it.

The census, re-measured, because the issue's numbers were stale

when filed today (3e86a7e)
columnar.h lines 957 1045
extern declarations 159 164
single-consumer functions 79 84

The header gained interface faster than #527 removed it. Of 137 function declarations, 84 have exactly one consumer outside the defining file and 49 are genuinely shared.

What moved, and what did not

The 22 functions defined in columnar_metadata.c and used by exactly one other file. The shared vocabulary they take — the Native*Metadata structs, the GUCs, the format constants — stays in columnar.h, which the new header includes. Nothing is made static, so #527's disqualification does not apply here: this relocates declarations, it does not narrow linkage.

Verified mechanically: each of the 22 appears exactly once in the new header and zero times in columnar.h.

A defect in my own measurement, since it decided which module went first

The first version of my census attributed PgColumnarBloomBuild and PgColumnarEncodeChunk to columnar_write_state.c, which defines neither. It picked whichever file matched first in directory order. Requiring a definition at column 0 — the style this tree uses, return type on its own line — changed the ranking:

module corrected broken tool said
columnar_metadata.c 22 outside the top three
columnar_reader.c 12 20
columnar_write_state.c 11 25, ranked first

The wrong module would have been extracted first. It was caught only because PgColumnarBloomBuild living in columnar_write_state.c is absurd on sight; two plausible filenames and I would have believed it.

Gate

major build warnings + errors
pg15a rc=0 0
pg16a rc=0 0
pg17a rc=0 0
pg18a rc=0 0
pg19a rc=0 0

make clean between each, per #536. Plus native_vecdecode 24/24 and harness_selftest 54/54 on pg18a.

The five-major build is the gate that matters here and I want to be explicit about why rather than pad the list: nothing in this diff alters a line of executable code. The failure mode of a declaration move is a consumer left without a declaration it needs, and that fails to compile — loudly, on whichever major first sees it — rather than misbehaving at run time. Zero occurrences of error: or warning: across five full builds is the direct evidence for that, and a suite run is corroboration rather than the proof.

Remaining

reader (12), write_state (11), storage (10), customscan (6), delete_vector (6) — one PR each. 84 declarations in a single diff is not a reviewable change, and each module's consumers are a different set of files.

I have not merged this and will not.

…mmandprompt#496)

columnar.h declares 137 functions. 84 of them have exactly one consumer outside
their defining file -- a private arrangement between two files that the other
twenty are forced to recompile for, and that anyone reading the header has to
scan past to reach the 49 that are genuinely shared.

This moves the largest single group: the 22 declarations defined in
columnar_metadata.c, into src/columnar_metadata.h beside it. The shared
vocabulary they take -- the Native*Metadata structs, the GUCs, the format
constants -- stays in columnar.h, which the new header includes.

The census in commandprompt#496 was stale and understated the problem. Re-measured on
3e86a7e:

                        when filed    today
    columnar.h lines           957     1045
    extern declarations        159      164
    single-consumer            79       84

so the header gained interface faster than commandprompt#527 removed it.

Re-measuring also caught an error in my own tool. Its first version attributed
PgColumnarBloomBuild and PgColumnarEncodeChunk to columnar_write_state.c, which
does not define either: it took whichever file matched first in directory order.
Fixed to require a definition at column 0, which is the style this tree uses, and
the ranking changed -- columnar_metadata went from outside the top three to the
largest module at 22, and columnar_write_state from 25 to 11. The wrong module
would have been extracted first.

Gate: builds on pg15a, pg16a, pg17a, pg18a and pg19a, make clean between each
(commandprompt#536), zero warnings and zero errors on all five. That is the gate that matters
for a declaration-only change: nothing here alters a line of executable code, and
a consumer left without the declaration it needs fails to compile rather than
misbehaving. native_vecdecode 24/24 and harness_selftest 54/54 on pg18a besides.

Five modules remain -- reader (12), write_state (11), storage (10), customscan
(6), delete_vector (6). One PR each: 84 declarations in a single diff is not a
reviewable change.

@ChronicallyJD ChronicallyJD left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed by checking the claims rather than reading the diff. The split is
clean and I have no objection to its content.
One structural finding, and it is
about #546 and #547 together rather than either alone.

Verified

Every new include is needed. For each .c that gained
#include "columnar_metadata.h", counted how many of the header's 22 symbols it
actually references:

columnar_metadata.c   22    columnar_vacuum.c        7
columnar_write_state.c 6    columnar_reader.c        3
columnar_projection.c  2    columnar_tableam.c       2
columnar_vector.c      1    columnar_visibilitymap.c 1

No include added that nothing uses, which is the usual way a mechanical split
adds churn.

The stated rationale holds, for all 22. The header says every declaration had
exactly one consumer outside its defining file. Counted per symbol across
src/*.c excluding columnar_metadata.c: not one of the 22 has more than one
consumer.
That is a checkable claim and it checks out, which is worth saying
because it is the entire justification for the split.

The finding: #546 and #547 are not stacked, and they collide

Both are titled as a chain — "the first of six", "the second of six" — but both
are single commits based on main, and neither contains the other.

Trial-merged them onto main in sequence:

merge 546 -> clean
merge 547 -> CONFLICT in src/columnar_reader.c
                        src/columnar_tableam.c
                        src/columnar_vector.c

Not columnar.h, as I first assumed — the removed hunks there are far enough
apart to auto-merge. It is the #include lines, added at the same place in the
consumers both PRs touch.

With six planned this compounds: each pair that shares a consumer collides the
same way, and the conflicts land in .c files rather than in the header being
split, so they read as unrelated to the change.

Either stack them (547 based on 546) or expect to rebase each one after the
previous merges. I have no view on which — I hit the auto-close trap stacking
#534 this morning and would not push anyone toward stacking without saying so.

Not a finding

A pure header split has no behaviour to assert, so "it builds and the suites
pass" is the right standard and CI covers it. I did not ask for a test.

@ChronicallyJD
ChronicallyJD merged commit d253bca into commandprompt:main Aug 10, 2026
11 checks passed
ChronicallyJD added a commit to ChronicallyJD/pgcolumnar that referenced this pull request Aug 10, 2026
…mandprompt#496)

The 12 declarations defined in columnar_reader.c with exactly one consumer
outside it: the fold entry points, the liveness cache, the projection and
filtered-read calls. Consumed only by customscan, tableam and vector.

Gate: builds on pg15a, pg16a, pg17a, pg18a and pg19a, make clean between each
(commandprompt#536), zero warnings and zero errors on all five. native_vecdecode 24/24,
native_fold_skipguard 4/4 and harness_selftest 54/54 on pg18a.

NOT independent of commandprompt#546, and I claimed it was. I measured the closest pair of
declaration removals in columnar.h between the two patches -- 49 lines, well
outside git's 3-line context -- and concluded they could both sit on main. The
gate applied both and refused:

    error: patch failed: src/columnar_reader.c:14
    error: patch failed: src/columnar_tableam.c:13
    error: patch failed: src/columnar_vector.c:36

The collision is not in the header at all. Both patches add their #include
immediately after #include "columnar.h" in the three files that consume BOTH
modules -- reader, tableam and vector -- so they edit the same anchor line. My
arithmetic was correct about the thing it measured and blind to the surface that
actually collides.

This still bases on main rather than stacking on commandprompt#546, deliberately. Stacking is
what auto-closed commandprompt#534 this morning when its base branch was deleted on merge, and
the conflict here is three adjacent include lines: whichever of the two lands
second wants a rebase that is mechanical and takes a minute. That is the cheaper
of the two failure modes.
ChronicallyJD added a commit to ChronicallyJD/pgcolumnar that referenced this pull request Aug 10, 2026
)

write_state (11), storage (10), customscan (6) and delete_vector (6) — the 33
remaining declarations with exactly one consumer outside their defining file.

ONE PR rather than four, and the reason is how commandprompt#546 and commandprompt#547 were reviewed: by
running checks — every new include used, no moved symbol having more than one
consumer — not by reading declarations. Correctness here is established by script,
so four diffs are no easier to review than one, and one diff collides with commandprompt#546
and commandprompt#547 once instead of up to fifteen times. Every pair of module PRs sharing a
consumer conflicts on the #include line, which is what made a six-PR series
expensive.

Not stacked, deliberately: a stacked PR auto-closes when its base is deleted on
merge (commandprompt#534, this morning). Merge in any order; whichever is second wants a
one-minute rebase of the include lines.

A defect in my splitting tool, found by the count assert and worth recording
because it silently produced a WRONG module assignment rather than an error:

columnar.h:353 is

    extern void PgColumnarEnsureStorageRow(Relation rel);\t/* pre-create ... */

which ends in "*/", not ";". The block scanner ran to the next line ending in ";"
and swallowed the declaration below it, filing PgColumnarReserveRowNumbers — a
storage symbol — under write_state.h. It compiles either way, because a
declaration is visible from whichever header the consumer includes; only the
"matched 9 of 10" assert on the NEXT module caught it. Terminator detection now
strips trailing comments.

I checked whether it reached the open PRs. It did not: commandprompt#546 and commandprompt#547 carry
exactly their intended sets, 22 and 12, nothing extra, nothing missing, nothing
left behind in columnar.h. That is luck — neither module's declarations happened
to sit beside one with a trailing comment.

All four verified exact: 11/11, 10/10, 6/6, 6/6, no extra symbol, none missing,
none stranded. columnar.h goes 1045 -> 997 lines and 164 -> 131 externs.

Gate: pg15a, pg16a, pg17a, pg18a, pg19a, make clean between each (commandprompt#536), zero
warnings and zero errors on all five. native_vecdecode 24/24, harness_selftest
54/54, replication 41/41 on pg18a.
ChronicallyJD added a commit to ChronicallyJD/pgcolumnar that referenced this pull request Aug 10, 2026
)

write_state (11), storage (10), customscan (6) and delete_vector (6) — the 33
remaining declarations with exactly one consumer outside their defining file.

ONE PR rather than four, and the reason is how commandprompt#546 and commandprompt#547 were reviewed: by
running checks — every new include used, no moved symbol having more than one
consumer — not by reading declarations. Correctness here is established by script,
so four diffs are no easier to review than one, and one diff collides with commandprompt#546
and commandprompt#547 once instead of up to fifteen times. Every pair of module PRs sharing a
consumer conflicts on the #include line, which is what made a six-PR series
expensive.

Not stacked, deliberately: a stacked PR auto-closes when its base is deleted on
merge (commandprompt#534, this morning). Merge in any order; whichever is second wants a
one-minute rebase of the include lines.

A defect in my splitting tool, found by the count assert and worth recording
because it silently produced a WRONG module assignment rather than an error:

columnar.h:353 is

    extern void PgColumnarEnsureStorageRow(Relation rel);\t/* pre-create ... */

which ends in "*/", not ";". The block scanner ran to the next line ending in ";"
and swallowed the declaration below it, filing PgColumnarReserveRowNumbers — a
storage symbol — under write_state.h. It compiles either way, because a
declaration is visible from whichever header the consumer includes; only the
"matched 9 of 10" assert on the NEXT module caught it. Terminator detection now
strips trailing comments.

I checked whether it reached the open PRs. It did not: commandprompt#546 and commandprompt#547 carry
exactly their intended sets, 22 and 12, nothing extra, nothing missing, nothing
left behind in columnar.h. That is luck — neither module's declarations happened
to sit beside one with a trailing comment.

All four verified exact: 11/11, 10/10, 6/6, 6/6, no extra symbol, none missing,
none stranded. columnar.h goes 1045 -> 997 lines and 164 -> 131 externs.

Gate: pg15a, pg16a, pg17a, pg18a, pg19a, make clean between each (commandprompt#536), zero
warnings and zero errors on all five. native_vecdecode 24/24, harness_selftest
54/54, replication 41/41 on pg18a.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant