Skip to content

test(scripts): pin QUICK_REFERENCE's Current Release rows by literal equality, not containment - #5385

Merged
os-support-ai merged 1 commit into
mainfrom
claude/issue-4913-quick-reference-equality
Aug 20, 2026
Merged

test(scripts): pin QUICK_REFERENCE's Current Release rows by literal equality, not containment#5385
os-support-ai merged 1 commit into
mainfrom
claude/issue-4913-quick-reference-equality

Conversation

@os-support-ai

Copy link
Copy Markdown
Collaborator

Fixes #4913

The defect

scripts/__tests__/quick-reference-current-release-4143.test.ts pins QUICK_REFERENCE.md's
"Current Release" rows to the manifests, and its header promises the forward direction in
as many words:

Forward: each anchored row must state what its manifest says. Bump the manifest without
touching the block and the row that lies goes red, naming the file to edit.

The forward assertions compared with toContain, which accepts any row carrying the
derived value as a substring — so every prefix superset of the anchor passed.
'^17.0.0-rc.6'.includes('^17.0.0') is true, which is why a doc left at the rc against
GA manifests kept both range rows green. Of the two directions, the broken one was the one
the header guaranteed; the reverse catch-all ("no un-derived literal") caught the rc.6
drift while answering a different question — new hand-written rows — and covered this one
only by accident.

The hole was the comparison, not the prerelease suffix. Measured below: ^17.0.0.1,
^17.0.0-beta and a ≥ 220 Node row against an engines.node floor of 22 all passed
the forward check identically. Every anchored row in the file used toContain, so every
anchored row had it.

The change (one file, scripts/__tests__/...4143.test.ts)

Per the 2026-08-19 triage ruling on the issue (option B, equality — not the
boundary-match fallback, which was not needed):

  • statedLiterals(row) extracts a row's version literals as whole tokens with the
    file's existing VERSION_LITERAL regex, which is lifted above both directions so the
    forward assertions and the reverse sweep cannot disagree about what a literal is.
  • expectRowStates(label, derived, message) compares that set to the derived set by
    equality. Comparing the whole set (rather than "must include") also means a row
    cannot smuggle a second, un-anchored literal in beside the right one, and an over-broad
    derivation fails loud instead of only widening what the block may say.
  • All seven forward assertions moved onto it — Version, Spec, Client, Node.js, pnpm,
    React, TypeScript. Fixing only the two rows the issue names (:212, :227) would leave
    the header's claim false for the other five, and the ≥ 220 measurement below shows
    they carried the same hole. Same defect class, same file, same gate, no new verification
    surface.
  • One toContain is kept deliberately, on the pnpm row, over the part that is not a
    version literal: equality holds the digits, containment holds the pnpm@ they belong
    to, so a row quoting the right version against the wrong package manager still fails.
  • The reverse catch-all is untouched. Two directions, two questions: forward asks
    whether this row still says what its anchor says, reverse asks whether anything here
    was never derived at all. A brand-new row is invisible to the first and caught by the
    second.
  • The header is rewritten to state what the forward direction now guarantees, the
    measurements behind it, and — a new section — where it still stops.

QUICK_REFERENCE.md is not touched. PR #4978 already brought the doc to the anchor,
and the doc is the subject the mutations below need.

Verification — the mutation is the acceptance test

Full before/after matrix, run from the repo root
(pnpm exec vitest run --reporter=verbose scripts/__tests__/quick-reference-current-release-4143.test.ts),
mutating QUICK_REFERENCE.md and restoring it between legs. "Before" is the same file
restored to origin/main. Every count was predicted before the run and every prediction
matched.

doc mutation before (toContain) after (equality)
(none — baseline) 8 passed 8 passed
Spec+Client ^17.0.0-rc.6 1 failed / 7 passed — reverse only 3 failed / 5 — Spec, Client, reverse
Spec+Client ^17.0.0.1 1 failed / 7 passed — reverse only 3 failed / 5 — Spec, Client, reverse
Spec+Client ^17.0.0-beta 1 failed / 7 passed — reverse only 3 failed / 5 — Spec, Client, reverse
Version 17.5.0-rc.1 1 failed / 7 passed — reverse only 2 failed / 6 — Version, reverse
Node ≥ 220 (floor is 22) 1 failed / 7 passed — reverse only 2 failed / 6 — Node, reverse
pnpm pin pnpm@10.31.0-rc.1 1 failed / 7 passed — reverse only 2 failed / 6 — pnpm, reverse
React 19.x.2 1 failed / 7 passed — reverse only 2 failed / 6 — React, reverse
Spec+Client ^17.0.0 or higher 8 passed 8 passed — see below

The rc.6 row reproduces the 1 failed / 7 passed reading recorded on the issue on
2026-08-17, and the forward assertion now names the row it is about:

AssertionError: QUICK_REFERENCE.md must state the spec range as exactly "^17.0.0":
expected [ '^17.0.0-rc.6' ] to deeply equal [ '^17.0.0' ]
- Expected
+ Received
  [
-   "^17.0.0",
+   "^17.0.0-rc.6",
  ]

Build artifacts between the edit and the thing under test: none. This is a node:fs
pin — the test reads QUICK_REFERENCE.md and the manifests off disk, imports no workspace
package, and no dist/ sits in the path. Nothing was rebuilt and nothing needed to be.

The one probe that stayed green, and why it is recorded rather than fixed

^17.0.0 or higher passes before and after — measured, not assumed. Both directions
reason about version literals, and that row states its anchor's literal and no other; the
extra words are prose. Closing it would mean writing the rows' wording into the test, and
the rows carry hand-written anchor notes precisely so they can be reworded. The boundary is
now recorded in the file header under "Where the forward direction still stops", so the
next reader knows where the guarantee runs out instead of inferring one that is not there.

Gates run at 28714a33b (the final commit; tree clean)

  • pnpm exec vitest run --maxWorkers=2 scripts/__tests__57 files, 1298 tests passed
  • pnpm run type-check:scripts (tsc -p tsconfig.scripts.json) — exit 0
  • pnpm exec eslint scripts/__tests__/quick-reference-current-release-4143.test.ts — exit 0
  • node scripts/check-control-bytes.mjs — OK, 4764 tracked text files
  • node scripts/check-changeset-presence.mjs — exit 0

Changeset: none, deliberately

check-changeset-presence.mjs reports "1 file(s) changed, 0 of them under the src/ of a
package the release covers … no changeset is owed."
The guarded surface is
the src/ of each package in the fixed group; scripts/ is not a workspace package and changesets
never versions it. The repo's own precedent for repo-level scripts/-only PRs is zero
changesets (#5084, #5167 — both test(scripts):, both one file, neither carrying one). The
empty-frontmatter form is used when the change lands inside a package and must declare
that it publishes nothing (e.g. anchor-drift-cumulative-report-4974.md, which touched
packages/*/src/__tests__/) — that is not this PR's shape.


Generated by Claude Code

…equality (#4913)

The forward direction of the objectui#4143 pin compared with `toContain`, so a row
passed whenever it merely CONTAINED its manifest value — every prefix superset of the
anchor went green. Measured on this file: with GA manifests and the doc left at
`^17.0.0-rc.6`, both range rows passed and only the reverse catch-all went red
(1 failed / 7 passed). `^17.0.0.1`, `^17.0.0-beta`, and a `≥ 220` Node row against an
`engines.node` floor of 22 behaved identically — so the hole was the comparison, not
the prerelease suffix, and it sat in every anchored row here rather than in the two
that happened to get caught.

Each anchored row's version literals are now extracted as whole tokens and compared to
the derived set by equality (`expectRowStates`). Under the same mutation the forward
assertions themselves go red and name the row: 3 failed / 5 passed.

The reverse sweep is untouched — two directions, two questions — and the file header
now states what the forward direction guarantees, plus where it stops.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RV6yuVCxymHYE16PL9vQkE
@github-actions github-actions Bot added the tests label Aug 20, 2026
@os-support-ai
os-support-ai marked this pull request as ready for review August 20, 2026 04:51
@os-support-ai
os-support-ai added this pull request to the merge queue Aug 20, 2026
Merged via the queue into main with commit a20b81d Aug 20, 2026
20 checks passed
@os-support-ai
os-support-ai deleted the claude/issue-4913-quick-reference-equality branch August 20, 2026 04:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

QUICK_REFERENCE Spec/Client row tests pass vacuously when the doc states a longer version than the manifest (toContain, not equality)

1 participant