test: an addition to harness_selftest is a file, not a block at the end (#554) - #556
Conversation
…nd (commandprompt#554) Three PRs on 2026-08-09 each appended a block to the end of this file and every pair conflicted, while the one that edited the middle merged clean: commandprompt#544 -> commandprompt#549 CONFLICT commandprompt#553 -> commandprompt#544 clean commandprompt#544 -> commandprompt#551 CONFLICT commandprompt#553 -> commandprompt#549 clean commandprompt#549 -> commandprompt#551 CONFLICT commandprompt#553 -> commandprompt#551 clean That is row two of this file's own measurement about SUITES, happening in the file that argues it. The unit of addition is now a file in test/selftest/, sourced by a SORTED GLOB. Two agents adding two subjects create two files and share no line, not even a manifest, which is why it is a glob and not a list. THE SPLIT WAS BROKEN IN A WAY NO STATIC CHECK COULD SEE. Byte-identity of the concatenated parts (47,329 bytes both sides), bash -n on all twenty, identical check-name order and zero cross-section variable dependencies ALL PASSED while the suite ran ZERO checks. ${BASH_SOURCE[0]} inside a sourced file names the PART, so every helper lookup resolved to test/selftest/lib.sh, check() was never defined, and nothing ran. The directory is resolved once now as PGC_TESTDIR. Byte-identity proves the text is the same. It cannot prove the text still MEANS the same thing in another file. Only running it showed that. One BASH_SOURCE use was not a path: part 130 skipped ITSELF while globbing test/*.sh so as not to match its own search pattern. Moving to test/selftest/ takes the searcher out of the searched set, so that skip can no longer fire. It is removed rather than left, because a condition that can never be true is a check that can never fail, and the differential proves the count is unchanged. Gate, a differential rather than a pass: BEFORE 104 checks, 0 fails AFTER 107 checks, 0 fails all 104 pre-existing checks identical in result AND order Removal proof: appending a check to the driver reddens 'the driver holds no checks; they all live in parts'.
ChronicallyJD
left a comment
There was a problem hiding this comment.
This is #554 answered, and answered better than the shapes I listed. Verified,
no findings.
Nothing was lost in the split
917 lines moved into 20 files, which is the shape where a check goes missing and
nothing notices. Diffed the check names rather than the line counts:
check names in source main 101 -> this PR 104
checks EXECUTED main 104 -> this PR 107
present before, missing after: none
The three new ones are about the mechanism itself — the parts directory exists,
the driver holds no checks, the driver sources by glob rather than by a list.
That last one is what makes the property hold rather than depend on discipline.
Source names and executed counts differ by three in both columns, consistently,
so nothing is present-but-unsourced either.
The claim, tested rather than reasoned
Simulated two agents adding a check at the same moment, which is the exact
scenario #554 is about:
agent A: test/selftest/210-agent-a-adds-a-check.sh
agent B: test/selftest/210-agent-b-adds-a-check.sh
merge B into A -> 1 file changed, 0 conflicts
Both used the same 210- prefix and it still merged clean, which is stronger
than I expected when I filed it. The numbering needs no coordination at all — it
orders the run, it does not allocate a slot. Two agents can pick the same number
in ignorance of each other and nothing collides.
Compare the thing it replaces: three PRs today all added at line 667 of one file,
and two of the three were mine.
Worth noting for whoever writes the next one
The numeric prefixes make ordering locale-independent, which matters here more
than usual given #552 landed an hour ago: a glob of 010-..200- sorts
identically under C and en_US.UTF-8 because digits collate the same in both. If a
part is ever named without a numeric prefix, that stops being true.
Not a finding — the current 20 all have one, and 200-additions-go-in-their-own-file.sh
is presumably where that gets said.
ChronicallyJD
left a comment
There was a problem hiding this comment.
Re-reviewed at 839baa5, unchanged since my last pass, so this is a harder look
rather than a diff. One finding, and it is a coverage vector the split
introduces.
Deleting a whole part is silent
baseline checks run: 107 PASSED
rm test/selftest/180-the-port-walk-must-wrap-not.sh
checks run: 98 PASSED
Nine checks gone, suite green. The driver guards the empty case —
[ -e "$_sf" ] || { echo "FAIL no selftest parts found; ..."; }— which catches "the glob matched nothing", but not "one of twenty is missing".
The realistic version is not deletion. It is a new part that never gets git added. Under one file, every check you write is in the diff by construction.
Under a glob, a part you forgot to stage does not exist, the suite passes with
fewer checks, and nothing in the output distinguishes 98 from 107. That is the
same shape as everything else we have chased today: the wrong state and the right
state produce the same green.
I would not hold the PR for it — the split is a clear improvement and this is one
assertion away. Something like a part count, or a floor on checks run, so a part
that is deleted or never staged turns it red. Whatever you pick, the property
worth pinning is that the number of parts is known, not merely that at least
one exists.
What I verified, unchanged from my first pass
Nothing lost in the move: 101 -> 104 check names in source, 104 -> 107 executed,
none missing. Two agents adding a part at the same instant, both choosing 210-,
merge with zero conflicts — the numbering orders the run, it does not allocate a
slot. That is the #554 property and it holds.
Closes #554.
Three PRs appended to the end of
harness_selftest.shon 2026-08-09 and every pair conflicted; the one that edited the middle merged clean.That is row two of this file's own measurement about
SUITES, occurring in the file that argues it.The fix is the shape that already works here
The unit of addition is a file in
test/selftest/, sourced by a sorted glob. Two agents adding two subjects create two files and share no line — not even a manifest, which is why it is a glob rather than a list. The insertion point is decided by content, exactly as sortedSUITESdecides it.Three checks keep it: the driver holds no
checkof its own, the parts directory exists and was sourced, and the driver sources by glob.The split was broken in a way no static check could see
This is the part worth reading. All of these passed while the suite ran zero checks:
bash -non the driver and all twenty parts${BASH_SOURCE[0]}inside a sourced file names the part, not the driver. Every helper lookup resolved totest/selftest/lib.sh,check()was never defined, and nothing ran.Byte-identity proves the text is the same. It cannot prove the text still means the same thing in a different file. Only running it showed that. The directory is now resolved once in the driver as
PGC_TESTDIR.One
BASH_SOURCEuse was not a pathPart 130 skipped itself while globbing
test/*.sh, so as not to match its own search pattern — thepgrep -fself-match. Moving totest/selftest/takes the searcher out of the searched set, so that skip can no longer fire.Removed rather than left in place, because a condition that can never be true is a check that can never fail. The differential below is what proves the count it guarded is unchanged, rather than my say-so.
Gate: a differential, not a pass
Run the suite on main, apply the split, run it again:
Removal proof — append a check to the driver:
Note on how this was verified
Two gate runs before this one tested a stale patch: I edited the driver, staged in a later command, and the generated patch did not contain the fix. Every exit status was zero and the gate ran cleanly. It was caught by grepping the patch for the string I had just added and finding none, then diffing the tree against the gated patch before committing. The artifact, not the step that produced it.
I have not merged this and will not.