test: the report must ask cb_verdict, not merely have it (#531 follow-up)

[ChronicallyJD]

#532 fixed the win count and added cb_verdict, cb_warm_spread and cb_band
with fifteen checks over their arithmetic. This adds the check nobody has: that
the report asks them.

The gap, proved by removal

test/bench_guards.sh sources bench/cb_guards.sh and never reads
bench/run_clickbench.sh, so the call site is invisible to it. Restoring the
#531 defect verbatim:

r1=$(ratio "$c" "$h")
if [ "$(awk -v r="$r1" 'BEGIN { print (r < 1) ? 1 : 0 }')" = 1 ]; then
        wins=$((wins + 1)); else losses=$((losses + 1)); fi

tree	suite
main	45 checks PASSED
#531 defect restored, main's suite	45 checks PASSED
#531 defect restored, this branch	49 checks, 2 FAIL
main, this branch	49 checks PASSED

The second row is the point. The fix was merged and the harness could have been
reverted to the exact defect it fixed without a single check reddening.

What the four checks are

Two premises and two assertions. ratio()'s output must reach printf and
nothing else, and the verdict must come from cb_verdict.

The premises are load-bearing rather than decoration. Rename r1 and a bare
"no ratio value decides anything" passes against a report that has gone back to
deciding on one under another name — the grep finds nothing, and nothing is what
it approves. So the suite first asserts that the report still captures ratio()
into a display variable at all.

These check source text, which is the weaker kind of check. They are here
because the behaviour they guard needs the 15 GB download and the tuned cluster
that bench_guards exists to do without. The alternative is not a better test;
it is no test, which is what the 45 amounted to at this seam.

They live in the suite rather than in bench/cb_guards.sh on purpose: that
file's contract is "Nothing here touches a database or the filesystem", and a
guard that reads a source file would break it.

Gate

bench_guards on main	45 PASSED
bench_guards on this branch	49 PASSED
removal proof (defect restored)	FAILED, rc=1 — non-vacuous
harness_selftest pg18a	54 PASSED
harness_selftest pg19a	54 PASSED

The pg19a number needed #536 dealt with first: the run initially failed every
cluster start because the build had reused pg18a's objects. make clean and a
rebuild for pg19a alone, and it passes. Nothing in that is caused by this branch
— the control on unpatched main fails identically — but it is why the gate took
two passes, and it is filed as #536.

Closes nothing on its own; it is the follow-up #532 wanted. See also #537 for
why diagnosing that gate failure took six probes.

[ChronicallyJD]

Reviewed by running it. The premise is right and the seam is the right one to
close. One finding, and it is the same class this PR exists to fix, one level
up: the check can be satisfied without the behaviour being present.

Verified

45 -> 49 checks, and it catches what it names. Restoring the #531 defect
verbatim in bench/run_clickbench.sh:

FAIL  ratio()'s output reaches printf and nothing else: got [2] want [0]
FAIL  and the verdict is asked of cb_verdict rather than recomputed inline: got [no] want [yes]

The premise on r[12]=$(ratio is the right one and it is doing real work: rename
the variable and the premise fails rather than the check silently approving.

The finding: a mention satisfies it, so the defect can return

grep -c 'cb_verdict' counts any occurrence, including one in a comment. I
removed the call, decided the verdict inline on $c and $h directly, and left
one comment line mentioning the name:

# a future refactor might reinstate cb_verdict here
if [ "$(awk -v x="$c" -v y="$h" 'BEGIN { print (x < y) ? 1 : 0 }')" = 1 ]; then
        wins=$((wins + 1)); else losses=$((losses + 1)); fi

Result: checks run: 49, bench_guards.sh: PASSED.

Both checks are bypassed at once, because deciding on $c and $h rather than
on $r1 also satisfies "ratio's output reaches printf and nothing else". So the
verdict is computed inline again, the report is back to #531's shape in
everything but the rounding, and the suite approves it.

This is not hypothetical in the way it might look. The likely path is not
somebody being clever, it is somebody refactoring the loop, leaving a TODO
mentioning cb_verdict, and the guard staying green.

The fix is one character

Grep for a call rather than a mention. Measured on the same bypassed tree:

loose  grep -c 'cb_verdict'    -> 1   approves
tight  grep -c 'cb_verdict "'  -> 0   rejects
and on the good tree, tight    -> 1   still approves

cb_verdict " is the shape a call takes here and a mention almost never does.
Same form as the call-site checks in #544, which land on pgc_start_log_report "
for exactly this reason.

What I could not fault

Both premises are load-bearing rather than decorative, and the "the alternative
is not a better test; it is no test" note is the right framing for admitting a
source-text check. The four checks are the correct four for this seam. This is a
one-character change from being airtight, and the seam it closes is one I left
open twice.

[ChronicallyJD]

Finding accepted and fixed at aaffd16. I reproduced your bypass before changing anything — it works exactly as you described.

Same tree, verdict recomputed inline on $c and $h, only a comment left mentioning the function:

	loose grep -c 'cb_verdict'	tight grep -c 'cb_verdict "'
bypassed tree	1 — approves	0 — rejects
good tree	1	1 — still approves

And you were right that both checks fall together: deciding on $c/$h instead of $r1 satisfies "ratio's output reaches printf and nothing else" at the same time, so the report returns to #531's shape in everything but the rounding and the suite says PASSED.

Now matched as a call. Re-gated:

baseline                     49 checks, PASSED
bypass applied (mention only) FAIL  and the verdict is asked of cb_verdict
                                    rather than recomputed inline: got [no] want [yes]
                              49 checks, FAILED

The comment carries the measurement and the realistic path you named — not somebody being clever, somebody refactoring the loop and leaving a TODO that mentions the function.

This is the same defect this PR exists to fix, one level up, in the fix itself. I wrote a check that a mention satisfies, in a PR whose argument is that a suite can approve a report which no longer does the thing. Worth stating plainly rather than quietly correcting: the seam was right and my assertion at it was not, and it took someone running the bypass to show that.