Read the suite list the way the runner reads it, and sort it (#469)

[ChronicallyJD]

Takes the item #469 asked for first: "Once #444, #446 and #468 land, I would like to do this first, before anything else goes in." All three landed.

It did not end where I expected. Two findings changed the shape of the fix, and both came from measuring rather than reasoning.

Finding 1: the bad resolution does not leave a stray command, it destroys the array

I had this recorded as "leaves a stray command that fails at run time with foo: command not found". That is wrong:

SUITES=(alpha beta gamma) stray_name
  -> stray_name: command not found
  -> ${#SUITES[@]} is 0

NAME=value cmd scopes the assignment to that one command, and an array literal is no exception. So the name after the paren does not join the array, it leaves SUITES unset and the matrix runs no suites at all. The runner's existing "NO SUITES RAN" guard is the backstop that stops that being a silent green.

Finding 2: the check that should have caught it was blind by construction

harness_selftest derived the list with awk '/^SUITES=\(/,/\)/' | tr | sed | grep, which is a reimplementation of bash's array parser. Against a runner carrying the mistake:

reader	sees
bash	0 suites
the awk parser	130 names, including the stray

So "every suite is registered in run_all_versions.sh" passed while the array was destroyed.

The runner now answers --list-suites, handled before the run lock because harness_selftest calls it from inside a running matrix and taking the lock there would refuse to answer. The gate asks the runner instead of parsing it, so there is one parser and no way for the two to drift. Verified by removal: restoring the awk parser turns the new checks red.

Finding 3: one name per line does not fix the conflicts

This is the part I would have shipped wrong. Measured by branching twice, adding a suite on each branch, and merging:

arrangement	two additions	result
one line (today)	same physical line	CONFLICT
one per line	both appended at the end	CONFLICT
one per line + sorted	names far apart	clean
one per line + sorted	names sort adjacently	CONFLICT

Everyone appends at the end. That is the shape all four of the conflicts in #469 had, and all four of mine the night #446/#468/#444 landed, so one-per-line alone would have left every one of them conflicting. Sorting is what gives a new suite an insertion point decided by its name.

The real-world shape, measured before and after:

BEFORE (main): two PRs each add a suite   CONFLICT
AFTER  (this): two PRs each add a suite   MERGES CLEANLY
AFTER: worst case, adjacent names         CONFLICT

Stated plainly: this is a large reduction, not a cure. Two suites whose names sort next to each other still collide. What has gone for good is the destructive resolution, because the closing paren is now on its own line and the gate catches that mistake whatever the layout.

A sortedness check keeps the property. Without it the order decays the first time somebody appends by hand and the reduction quietly disappears.

What changes for readers

Run order. The array is walked in order, so the summary prints alphabetically and ports are assigned in that order. Neither is a correctness property: the parallel batch already runs six at a time with no ordering guarantee, and runs_alone still selects the sequential ones. The set of suites is unchanged, verified through --list-suites before and after rather than by reading the diff, which is the one thing this file's history says never to trust: 129 before, 129 after, identical.

Not taken

Generating the list from test/*.sh with an exclude list, the other option #469 offered. It would end the conflict completely, but it converts "a stray .sh in test/ turns the gate red until you decide what it is" into "a stray .sh is silently gated". Recorded so the choice is visible rather than silently taken.

Gate

Full five-major matrix, because this changes how every suite is scheduled. Results to follow in a comment.

[ChronicallyJD]

Matrix: ALL VERSIONS PASSED

PASS  PG15  (126 ran, 3 skipped)
PASS  PG16  (126 ran, 3 skipped)
PASS  PG17  (126 ran, 3 skipped)
PASS  PG18  (127 ran, 2 skipped)
PASS  PG19  (129 ran, 0 skipped)
versions run: 5 of 5 configured
### m473 exit=0 ###

Zero FAIL lines. harness_selftest=PASS on all five majors. The only skips are the version-gated ones (native_repack, pg19_vacuum_options, temporal).

Confirmed the tree under test actually carried the change rather than trusting the green: the build tree has the sorted array, --list-suites, and the caching fix, and harness_selftest run there reports the full 40 checks.

The first matrix caught a real defect in this PR

Worth recording, because it is the failure class this PR exists to remove and I had reintroduced it.

The first run failed on PG16 and PG17 only, while PG15/18/19 passed:

FAIL  every suite is registered in run_all_versions.sh:
      got [unregistered: differential parallel_copy parallel_vector_agg planner_choice_quality]
      want [none]

All four are registered, and the names were different on each major. That is load, not a missing entry.

listed_suites forked bash run_all_versions.sh --list-suites on every call, and the two checks below it call it once per test file: roughly 250 forks inside a six-way parallel matrix. A transient failure to fork returns an empty list, and an empty list reads as "this suite is unregistered", naming whichever files happened to be in hand.

So an intermittent red, against innocent suites, caused by the gate rather than the code. Fixed by asking once and caching, with an empty answer now a named premise instead of a silent cause. The fixture path still takes an explicit argument, since it parses a different file.

Two things this depended on: keeping the whole matrix log instead of piping it through tail (the previous driver discarded exactly this kind of detail), and running the full matrix rather than the two-major cadence, since the failure only appears under concurrency.

Leak check

The matrix left 0 orphaned postmasters, which is #446 and #471 holding.