test(scripts): pin QUICK_REFERENCE's Current Release rows by literal equality, not containment - #5385
Merged
Conversation
…equality (#4913) The forward direction of the objectui#4143 pin compared with `toContain`, so a row passed whenever it merely CONTAINED its manifest value — every prefix superset of the anchor went green. Measured on this file: with GA manifests and the doc left at `^17.0.0-rc.6`, both range rows passed and only the reverse catch-all went red (1 failed / 7 passed). `^17.0.0.1`, `^17.0.0-beta`, and a `≥ 220` Node row against an `engines.node` floor of 22 behaved identically — so the hole was the comparison, not the prerelease suffix, and it sat in every anchored row here rather than in the two that happened to get caught. Each anchored row's version literals are now extracted as whole tokens and compared to the derived set by equality (`expectRowStates`). Under the same mutation the forward assertions themselves go red and name the row: 3 failed / 5 passed. The reverse sweep is untouched — two directions, two questions — and the file header now states what the forward direction guarantees, plus where it stops. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RV6yuVCxymHYE16PL9vQkE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4913
The defect
scripts/__tests__/quick-reference-current-release-4143.test.tspins QUICK_REFERENCE.md's"Current Release" rows to the manifests, and its header promises the forward direction in
as many words:
The forward assertions compared with
toContain, which accepts any row carrying thederived value as a substring — so every prefix superset of the anchor passed.
'^17.0.0-rc.6'.includes('^17.0.0')istrue, which is why a doc left at the rc againstGA manifests kept both range rows green. Of the two directions, the broken one was the one
the header guaranteed; the reverse catch-all ("no un-derived literal") caught the rc.6
drift while answering a different question — new hand-written rows — and covered this one
only by accident.
The hole was the comparison, not the prerelease suffix. Measured below:
^17.0.0.1,^17.0.0-betaand a≥ 220Node row against anengines.nodefloor of22all passedthe forward check identically. Every anchored row in the file used
toContain, so everyanchored row had it.
The change (one file,
scripts/__tests__/...4143.test.ts)Per the 2026-08-19 triage ruling on the issue (option B, equality — not the
boundary-match fallback, which was not needed):
statedLiterals(row)extracts a row's version literals as whole tokens with thefile's existing
VERSION_LITERALregex, which is lifted above both directions so theforward assertions and the reverse sweep cannot disagree about what a literal is.
expectRowStates(label, derived, message)compares that set to the derived set byequality. Comparing the whole set (rather than "must include") also means a row
cannot smuggle a second, un-anchored literal in beside the right one, and an over-broad
derivation fails loud instead of only widening what the block may say.
React, TypeScript. Fixing only the two rows the issue names (
:212,:227) would leavethe header's claim false for the other five, and the
≥ 220measurement below showsthey carried the same hole. Same defect class, same file, same gate, no new verification
surface.
toContainis kept deliberately, on the pnpm row, over the part that is not aversion literal: equality holds the digits, containment holds the
pnpm@they belongto, so a row quoting the right version against the wrong package manager still fails.
whether this row still says what its anchor says, reverse asks whether anything here
was never derived at all. A brand-new row is invisible to the first and caught by the
second.
measurements behind it, and — a new section — where it still stops.
QUICK_REFERENCE.mdis not touched. PR #4978 already brought the doc to the anchor,and the doc is the subject the mutations below need.
Verification — the mutation is the acceptance test
Full before/after matrix, run from the repo root
(
pnpm exec vitest run --reporter=verbose scripts/__tests__/quick-reference-current-release-4143.test.ts),mutating QUICK_REFERENCE.md and restoring it between legs. "Before" is the same file
restored to
origin/main. Every count was predicted before the run and every predictionmatched.
toContain)^17.0.0-rc.6^17.0.0.1^17.0.0-beta17.5.0-rc.1≥ 220(floor is 22)pnpm@10.31.0-rc.119.x.2^17.0.0 or higherThe rc.6 row reproduces the 1 failed / 7 passed reading recorded on the issue on
2026-08-17, and the forward assertion now names the row it is about:
Build artifacts between the edit and the thing under test: none. This is a
node:fspin — the test reads QUICK_REFERENCE.md and the manifests off disk, imports no workspace
package, and no
dist/sits in the path. Nothing was rebuilt and nothing needed to be.The one probe that stayed green, and why it is recorded rather than fixed
^17.0.0 or higherpasses before and after — measured, not assumed. Both directionsreason about version literals, and that row states its anchor's literal and no other; the
extra words are prose. Closing it would mean writing the rows' wording into the test, and
the rows carry hand-written anchor notes precisely so they can be reworded. The boundary is
now recorded in the file header under "Where the forward direction still stops", so the
next reader knows where the guarantee runs out instead of inferring one that is not there.
Gates run at
28714a33b(the final commit; tree clean)pnpm exec vitest run --maxWorkers=2 scripts/__tests__— 57 files, 1298 tests passedpnpm run type-check:scripts(tsc -p tsconfig.scripts.json) — exit 0pnpm exec eslint scripts/__tests__/quick-reference-current-release-4143.test.ts— exit 0node scripts/check-control-bytes.mjs— OK, 4764 tracked text filesnode scripts/check-changeset-presence.mjs— exit 0Changeset: none, deliberately
check-changeset-presence.mjsreports "1 file(s) changed, 0 of them under the src/ of apackage the release covers … no changeset is owed." The guarded surface is
the
src/of each package in thefixedgroup;scripts/is not a workspace package and changesetsnever versions it. The repo's own precedent for repo-level
scripts/-only PRs is zerochangesets (#5084, #5167 — both
test(scripts):, both one file, neither carrying one). Theempty-frontmatter form is used when the change lands inside a package and must declare
that it publishes nothing (e.g.
anchor-drift-cumulative-report-4974.md, which touchedpackages/*/src/__tests__/) — that is not this PR's shape.Generated by Claude Code