test(docs): check the evidence anchors against source - #802
Conversation
|
CI on the first push confirms the hold is the only thing wrong with this
So |
|
The stated blocker is cleared: v0.24.0 published on 2026-08-22, tag Worth knowing before you rebase, because the reasoning in the description is close but not quite the mechanism. What protects the archive is not the tag existing. #799 was re-landed on that basis in #806, with Two things to carry over when you take this off draft:
Ordering note: this and #806 both touch docs content, so whichever lands second wants a rebase. |
74decb7 to
debfe23
Compare
|
Unblocked. v0.24.0 published 2026-08-22T09:35:12Z, the release carries Merge #806 first. It re-lands the reverted #799 and shares two files |
|
Green on the rebase. All 15 non-skipped checks pass, including
Two things a reviewer should not read as oversights, both deliberate:
Still blocked on #806 for merge order, per the conflict note above. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: debfe23dd4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Added It states what the check rejects, that a drifted citation is now a merge Documenting this in release notes instead would have reached the wrong |
debfe23 to
57f6cb7
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 57f6cb76f7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Four Codex findings all confirmed against the source and fixed, with a test
The lowerCamelCase fix immediately found two real anchors, both quoting a
Both anchors gained the path carrying the key rather than losing the claim. That same Local: Note for #806: this adds |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2caf7b991e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
2caf7b9 to
1c1b728
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1c1b728863
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c6daf04756
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ebbfefeee0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ebbfefe to
e3a36b6
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e3a36b68cf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 74a6d5b945
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 50329626a5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
5032962 to
6802078
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6802078110
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Every factual claim the docs site makes about a source repository is
anchored in a {/* Evidence: ... */} comment naming the code, test, or
contract that backs it, and nothing verified those anchors. Across three
recent pull requests, 24 stale path:line-range citations and one citation
naming a test that does not exist were found only by reading them.
The checker resolves the paths an anchor cites, including the
continuation forms (src/foo.rs and a bare sibling filename read against
the previous path, and a bare :N-M read against the file before it), then
requires that every cited path exists, that every line reference is
inside its file, and that every symbol-shaped token the anchor names
occurs in at least one path that anchor cites.
Line-range citations are counted but not yet refused. 81 remain in the
tree; --strict-line-refs turns them into failures once they are converted
to symbol citations.
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
What the anchor checker found, plus one claim the same reading exposed. The file secret provider accepts mode 0400 or 0600, not 0600 alone: validate_file_metadata() tests matches!(mode, 0o400 | 0o600), and evidencectl writes a 0400 runtime, so an operator reading "rejects anything else" would expect a working deployment to be refused. Three security pages claimed the stricter rule. The security landing page also cited file_secret_uses_open_file_owner_and_exact_mode_checks, which exists nowhere in the workspace; the real test is file_secret_accepts_only_owner_read_and_optional_owner_write_modes. The threat model carried both errors and was corrected in #806. The DPI safeguards page attributed Relay's problem catalog to crates/registry-relay-v2/src/problem.rs, which holds only the ProblemCodeResponseExt trait, and credited it with a PROBLEM_BASE constant that exists only in Evidence. The 26 entries live in registry-relay-http-contract and each carries its own literal type_uri. The count was right; the crate and the constant were not. The same anchor's contract.rs range named PurposeConstraint and AuthorityRowBinding but pointed at DatePrecision and DisclosureProfile, about 160 lines away, so that range and the stale auth.rs range beside it are dropped rather than repaired; the anchor already names both symbols. Security-sensitive review note: documentation only, no runtime or contract change. Every corrected claim was re-verified against the runtime source named in its anchor. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Review found four ways the check reported success where it should have reported drift, which is worse than no check because it manufactures confidence. A continuation path with nothing before it, such as src/data/projects.yaml, resolved under docs/site but was optional, so deleting the file removed the citation from validation instead of failing. It is a definite path and is now required. No real anchor relied on the leniency: the cited-path count is unchanged. Line references were only checked against the end of the file, so :0, a descending range such as :5-3, and a range starting past EOF all passed. A range now has to start at line 1 or later, end at or after its start, and lie inside the file. extractSymbols read SCREAMING_SNAKE_CASE, snake_case, and UpperCamelCase but not lowerCamelCase, so the wire and configuration keys the docs quote were never checked at all. Misspelling packageRevision left the gate green. Citations could leave the repository. The path pattern matches .. segments and the resolved path was never tested for containment, so an anchor citing crates/../../../etc/passwd counted as a valid existing path. Traversal is now an error rather than a skip, since skipping is how it would hide. Security-sensitive review note: the traversal fix is the reason this is not purely a correctness change. The check reads and stats every path it resolves, so before this it would follow a citation to an arbitrary file outside the tree. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The lower-camel-case check finds two anchors naming a wire key while citing only the Rust file, which carries the snake_case spelling. records-stay-home.mdx quotes authorityRowBinding and cites contract.rs, where the enum is AuthorityRowBinding and the field is authority_row_binding; the key itself is in the relayctl authoring schema. verify/index.mdx quotes recordsEquivalentTo and etagSameAs and cites fixtures.rs, which spells them records_equivalent_to and etag_same_as; both keys are in the business-registry acceptance expectations. recordsEquivalentTo passed only because a test literal in fixtures.rs happens to contain it, so the pair was inconsistent for an incidental reason. Both anchors gain the path carrying the key rather than losing the claim: the prose is describing what an adopter writes, so the spelling it quotes is the right one. The same records-stay-home anchor cited contract.rs:947-976 for AccessRule and AuthorityRowBinding, which are at 1107 and 1134; that range now holds PartialStringReveal, DateInputType, DatePrecision, and DisclosureProfile. The range is inside the file, so no line check can catch it. Dropped, since the anchor already names the symbols. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Six cases the checker claimed to check and silently did not: - a qualified symbol was read from its last segment only, so a typo in the module, type, or enum that qualified it stayed green; every segment that carries a symbol shape is now checked on its own. - an identifier spelled with empty parentheses, router() or prepare(), matched none of the four case shapes and was dropped; the anchors already spell a function reference that way, so it is read as a name whatever its case. - a bare filename the repository keeps at its root resolved against no candidate at all, so its symbols were never looked for. The root is tried after the search inside the cited unit rather than before it, so a crate's own README wins over the workspace one instead of losing to it. - a sibling filename read against the last full path rather than the last cited one, so a continuation between them was ignored. Only a citation that claims a repository path moves that anchor: a sibling is a reading of the prose and leaves it where it is. - a bare .rhai script was not a sibling filename, though the scripts are cited as bounded request preparation and source extraction. - a line suffix cut short, :5-, parsed as the single line 5 and is now reported as the malformed reference it is. The docs need no correction under any of them: 25 more symbols are checked, and every one resolves in a path its anchor already cites. The cited-path and line-range counts are unchanged, which is what holds the sibling-anchor change honest: moving that anchor too eagerly drops a path from the count in silence. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The lexical containment check refused a `..` segment and resolved the citation against the repository root, but never consulted a real path. A tracked symlink pointing outside the checkout passed it, and both the stat and the read then followed the link, so content the repository does not hold could satisfy an anchor. Resolve both sides before deciding: the root as well, since a macOS temporary directory is itself reached through a symlink. A path with no real path stays the missing citation it already was. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
An exact wire value such as ES256 or RS256 matched none of the four case shapes, so an anchor could name one that no cited path holds and stay green. Add a fifth shape for it: uppercase letters and digits only, with at least two letters and at least one digit. Both exclusions are measured, not guessed. Requiring a digit keeps out the acronyms the prose spells in capitals, JSON, HTTP, SQL, and the second letter keeps out the product version words V1 and V2; without them the rule reports correct anchors today. Mixed-case values such as EdDSA stay uncaught: the only shape that admits them also admits OpenAPI, OpenID, OpenCRVS and 25 more prose names, one of which already resolves only through an unrelated file. Cited symbols checked go from 286 to 288, with no change to the cited paths or line ranges. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The guarantee the page advertises has to match what the check does, or it converts an unchecked claim into a checked-looking one. Name the shapes a symbol is read by, and name the one gap: an all-capital wire value with no digit is prose to the check. Reaching EdDSA needs a shape that also admits OpenAPI, OpenCRVS, and every acronym in the prose, so it stays uncaught by choice rather than by oversight. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
An anchor names several files in one directory in a compact brace form,
`src/{contract,compiler,api}.rs`. The citation pattern held no brace, so the
match stopped at the directory before it and the check validated that
directory alone: renaming or deleting any file the list names left the gate
green.
Expand the list instead of refusing it. A brace segment is read where a path
segment can start, so every entry becomes its own full or continuation
citation with the suffix the entries share, resolved and counted on its own. A
brace group the prose itself writes, `{ claim, allowed }`, sits after no slash
and stays prose.
The one brace list in the content names six files, so the cited-path figure
moves from 333 to 338: six citations in place of the single directory the
match used to stop at. Anchors, symbols, and line ranges are unchanged.
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
An anchor names configuration and wire keys as dotted paths, such as evidence_data_request.transport_absences.credentials. The word pass splits on the dot and then drops any segment that carries no symbol shape, so the leaf of every such key went unchecked: misspelling it left the gate green. Read a dotted token as one key path and check every segment of it, once one segment already carries a symbol shape. That mark is what was measured, not guessed. Checking every segment of every dotted token adds 73 checks over 30 distinct tokens and reports five of them, all from the version string v0.9.0, and pulls in org, registrystack, and a bare digit besides. Requiring one shaped segment adds 15 checks over 11 distinct tokens with nothing reported, and none of the 15 passes by coincidence: each resolves in a file that also holds every shaped segment of the same token, which for request.fields_invalid and classification.context.selector_more_restrictive_than_disclosure is the file holding that exact literal. A `*` is skipped rather than looked up, since it stands for any key rather than naming one. One gap stays open by choice. A key path no segment of which carries a shape, sources.*.authentication.kind, is still unread. Treating the `*` itself as the mark would reach it and reports nothing today, but the three segments it would check are among the commonest words in the tree, with no shaped segment to tie the match to the file the sentence is about, so it would look checked while biting on almost nothing. Cited symbols move from 288 to 303, the 15 the measurement predicted. Anchors, paths, and line ranges are unchanged. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The sibling form needed a path before it, so a filename in first position produced no citation at all and the file it named never reached the symbol check. Read it against the repository instead: the file kept at the root, then a single unambiguous file of that name in the tree, the two fallbacks the sibling form already had. It stays a reading of the prose rather than a path claim, like every other bare filename: with no path before it, nothing says the repository owns the name, so an adopter's metadata.yaml is still left alone. The one anchor that opened on a filename, contracts.yaml in the boundaries map, names docs/site/src/data/contracts.yaml, whose registry-manifest.metadata-yaml entry carries the consumer note it cites. Spell that path out: an explicit path is reported when it goes missing, where a bare name would quietly stop resolving. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The sibling form asked for a filename extension, so a bare directory that continued a cited directory carried no citation and renaming it left the gate green. Read a trailing-slash name against the directory cited before it and report it when it is gone, which is what the one real case wanted: protected-read-evidence/ under the reference deployment projects. A trailing-slash name after a filename stays prose. governed/ and generated/ beside package.rs name directories relayctl writes into a package, not directories the repository holds, so checking them would report correct anchors. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
UpperCamelCase asked for an [A-Z][a-z]+ boundary at every word, so a name whose capitals run together went unchecked while the guidance said the shape was read. Seven declared types in crates/ carry it, all on the OAuth token surface, and the anchors that would cite them are the ones a rename there would drift. The discriminator is two lower-case runs and one capital run of two or more. Measured over all 134 anchors it adds no symbol check and no distinct token: OpenAPI, SQLite, OpenCRVS, and EdDSA carry one lower-case run, SDMX and JWKS carry none, so the acronyms the prose is full of stay prose. A name carrying one run, SDMXProfile, stays unchecked with them, and the guidance now names that boundary with the other gaps. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The seven readings build object literals of different shapes, so the field a later reading looks for reads as absent on the ones that do not carry it. A typedef states which fields every citation has and which belong to one reading. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The anchors cite source across the whole workspace, but the only job that ran them sat behind the changed-path classifier's docs allow-list, which names five crate files. A rename of any other cited source left the docs job skipped and the drift merged green. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
UpperCamelCase asks for two capitalized chunks, so a one-word name such as Public, Snapshot, or Visibility was read as prose and left unchecked. The qualified spelling is already checked segment by segment, so naming the owning type brings each variant under the existing rule. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
A sibling stays optional so an anchor may name a file the repository does not own, but that also let a repository-owned file be deleted with the check still green. Only this repository writes Rust into the stack, so a bare .rs name is a claim: measured over every anchor, the rule requires six of the seven siblings that resolve today and fails none of the three that do not. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The escape check reads a citation's candidate list, and the file kept at the
repository root is no member of it: a bare filename carries it separately as
its last resort. A root file that is a symlink pointing outside the checkout
therefore passed the check, resolved the citation, and had its outside
contents read to satisfy the anchor's symbols.
Unwind the resolution chain into explicit steps so the root candidate can be
refused where it is consulted, and report the refusal rather than skipping the
citation: it is the only thing that would have resolved, so falling through
silently would leave the anchor unchecked.
The search for a single unambiguous file of the same name needs no such check.
It walks the tree with readdirSync(..., { withFileTypes: true }), whose
Dirent.isFile() and isDirectory() read the entry rather than its target, so a
symlink is neither and never enters the walk.
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The citation roots named six directories and the repository keeps ten, so a path into editors/, docker/, or .cargo/ matched no citation pattern, parsed as prose, and left its anchor checked against nothing. A rename under any of them was invisible to the gate. Name all ten, and add a test that reads the tracked directories out of the committed tree so the next one the repository grows fails the gate rather than slipping through it. Git is what says which directories are tracked: a listing of the checkout also carries build output and local tooling, and which of those are present differs between a clean CI checkout and a working machine. schemas/ is on both root lists, because a crate or product keeps one and so does the repository. Reading it as a repository path outright would take the continuation reading away from the two dozen crate and product schemas directories in exchange for the one file the repository keeps at the top, so a citation that starts there keeps the continuation candidates and gains the repository reading as its last one. The nearer directory still wins, as it does for a bare filename that may name a file kept at the root. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
An anchor whose paths all failed to be read as citations, by a typo such as crate/ for crates/ or by naming a directory the roots do not cover, reached the symbol check with nothing to read against and was skipped. It counted as an anchor and contributed no checked path and no checked symbol, so the gate reported a guarantee it had not made for it. The two ways an anchor arrives there are different mistakes and are told apart. One that parsed no citation at all cites no path in this repository: the writer named nothing the check could open, and pairing an upstream standard with the file that implements it is the fix. One that parsed citations none of which resolved is reported as resolving none of them: every path it named was read as prose, which is what a bare filename the repository does not own is, and the anchor needs one path the repository does hold. An anchor whose citations were already reported keeps its own message, because it fails on those and a second line names no further drift. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The citation pattern reads a valid line suffix and stops, so whatever the anchor wrote past it was thrown away. `:abc` left the citation checked as the bare file, and `:1foo` left it checked as line 1, neither of them a line the anchor meant, and neither reported. Only the range cut short, `:5-`, was caught, and only because a trailing hyphen was looked for by name. Read the leftover instead: word characters or a hyphen, which a well-formed reference would have carried itself, optionally behind the colon that opens one. That is what a mis-spelled reference leaves and what a correct one never does, since prose puts a space after a colon it writes and puts its punctuation after a reference it ends on. The existing report carries the leftover rather than a hardcoded hyphen, so the three shapes read alike. The rule fires on none of the 134 anchors in the tree, which keeps the counts and the zero errors exactly where they were. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Whether a trailing-slash name continues a directory or is prose was decided by looking for a dot in the last segment of the path before it. An extensionless executable such as release/scripts/registry-release has no dot, so it read as a directory and the name after it was resolved beneath a file, which can never exist. A correct anchor for any of the repository's extensionless scripts was reported as drift. The parse stays free of the filesystem, which is what makes it directly testable: it records the path the child continues, and the resolution, which already opens the tree, reads that path's kind and drops the citation where it is not a directory. A name whose parent is a file, or is nothing the repository holds, is prose and is left alone rather than reported. This is the same discrimination that already made governed/ and generated/ correct beside package.rs in explanation/publishing-pipeline.mdx. The dot in that filename is what saved them; the parent's real kind saves them for the reason they are actually prose, and that anchor is the regression test in the tree. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The five fixes changed what an anchor may say and what the gate promises, so the guidance says it: an anchor has to cite at least one path the repository holds, a citation may start at any top-level directory, `schemas/` reads nearest first because a crate and the repository both keep one, a path a symlink leads out of the checkout is refused, a line reference is spelled `:12` or `:12-14` and nothing else, and a trailing-slash name continues the path before it only where the repository holds that path as a directory. The bare filename read as prose keeps its exception and gains its condition: it is fine wherever the anchor resolves some other path. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
6802078 to
73f6c95
Compare
A bare child continued the parse's first candidate rather than the path the citation actually resolved to. Where a shared root resolved through a fallback, the repository's own schemas/ rather than the cited crate's, that first candidate named nothing, so the guard read it as prose and dropped the child citation without checking or counting it: deleting the directory it named left the gate green. The parse now records the child's bare name and the resolution reads it against the directory the anchor reached, dropping that position wherever a citation resolves nothing so a child after a reported miss stays prose instead of being read against a stale directory. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 73f6c959a9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Confirmed and fixed in 0003231. The defect is exactly as described. What changed:
Test, written before the fix and failing on the parent commit Suite is 78/78, up from 77. Worth stating plainly: running the gate over the One related thing left alone, since it is a different shape of change: the parse |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 000323199b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A line reference the anchor carried on past the line the pattern read, the `.5` of `:1.5`, was thrown away and the citation checked against line 1. It is read now, and only where the token carried a reference, so the full stop that ends a sentence on a path stays punctuation. A bare JavaScript or TypeScript filename was no sibling the grammar knew, so a name that resolves was never counted and `projectRoot.ts` was read as a dotted key path instead, demanding a symbol spelled `ts`. Both are siblings now. Whether one has to resolve is unchanged: only a Rust name does, because only this repository writes Rust into the stack. A file with no extension is a script the repository keeps, so a symbol an anchor cites from `release/scripts/registry-release` was reported absent wherever the anchor cited the directory holding it. The directory search reads it now. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
1 participant
Do not merge until v0.24.0 publishes
This changes docs content, so it moves the built v0.24.0 docset and would
fail
archive:snapshot v0.24.0 --verify-lockexactly as #799 did befored1a6c7b reverted it. v0.24.0 has a recorded lock entry but no tag and no
release yet. Draft until it publishes.
The checker cannot be split out and landed first: on current
mainitfails on the phantom test name and on
PROBLEM_BASE, so the gate and thecorrections have to land in the same commit range or
maingoes red.What this adds
Every factual claim the docs site makes about a source repository is
anchored in a
{/* Evidence: ... */}comment naming the code, test, orcontract behind it. Nothing verified those anchors. Across three recent
pull requests, 24 stale
path:line-rangecitations and one citationnaming a test that does not exist were found only by reading them by hand,
and each one turned out to be one instance of a multi-instance defect.
docs/site/scripts/check-evidence-anchors.mjsresolves the paths ananchor cites, including the continuation forms the prose uses (a bare
src/foo.rs, a bare sibling filename, and a bare:N-Mall read againstthe path before them), then requires:
that same anchor cites.
A token that resolves to no repository path is treated as prose naming a
file the repo does not own and is skipped, so the check reports drift
rather than vocabulary. Path tokens are stripped before symbol
extraction, so a filename stem is never read as a symbol.
Current tree:
121 anchors, 298 cited paths, and 204 cited symbols checked; 81 line-range citations found. Line-range citations are countedbut not yet refused;
--strict-line-refsturns them into failures oncethe remaining 81 (49 distinct targets across 11 pages) are converted to
symbol citations. That conversion is deliberately not in this PR.
Wired into
check:sourceimmediately aftercheck:evidence-links.19 tests in
check-evidence-anchors.test.mjs,node:test, temp-repofixtures, no dependencies.
What it found
The file secret provider accepts
0400or0600, not0600alone.validate_file_metadatatestsmatches!(mode, 0o400 | 0o600), andevidencectlitself writes a0400runtime file, so an operator reading"the file provider rejects anything else" would expect a working
deployment to be refused. Four pages carried the stricter claim:
explanation/threat-model.mdx,security/index.mdx,security/hardening-checklist.mdx,security/evidence.mdx.Prescriptive guidance that tells an operator to write
0600is leftalone; that is sound advice, not a false claim about the validator.
file_secret_uses_open_file_owner_and_exact_mode_checksdoes notexist anywhere in the workspace. Two pages cited it. The real test is
file_secret_accepts_only_owner_read_and_optional_owner_write_modes.Relay's problem catalog was attributed to the wrong crate.
explanation/dpi-safeguards-alignment.mdxcitedcrates/registry-relay-v2/src/problem.rs:14-45, which holds only theProblemCodeResponseExttrait, and credited it with aPROBLEM_BASEconstant that exists only in Evidence. The 26 entries are macro-generated
in
registry-relay-http-contractand each carries its own literaltype_uri. The count was right; the crate and the constant were not.Two
contract.rsline ranges pointed about 160 lines off target,naming
PurposeConstraintandAuthorityRowBindingbut landing onDatePrecisionandDisclosureProfile. Both anchors already name thesymbols, so the line numbers are dropped rather than repaired.
last_reviewedbumped on the six edited pages.Review note (security-sensitive surface)
Documentation only. No runtime, contract, schema, or generated artifact
changes. The corrections touch how secret-file permission validation and
Relay's problem catalog are described, both security-sensitive surfaces,
so every corrected claim was re-verified against the runtime source its
anchor names rather than against another doc page.
Verification
From
docs/site/:npm test: 383 pass, 0 failnpm run check: green, includingcheck-llms561 passed / 0 failed,SEO over 396 pages, and 18439 internal links and assets
npm run check:evidence-anchors: greenThe checker was mutation-tested rather than taken on trust: a phantom
symbol, an out-of-range line reference, and a missing path were each
injected and each diagnostic confirmed to fire.
Not folded in
Re-landing #799 is left as its own act, so the revert in d1a6c7b keeps
a clean audit trail. Say the word if you would rather both ride together
once v0.24.0 publishes.