diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e79b0f41e..497aec28f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -893,6 +893,30 @@ jobs: rust:1.95-trixie@sha256:f49565f188ee00bc2a18dd418183f2c5f23ef7d6e691890517ed341a598f67c3 \ bash /work/docs/site/scripts/check-evidence-tutorials.sh + evidence-anchors: + # The documentation anchors cite source across the whole workspace, so this check + # runs on every pull request rather than behind the changed-path classifier: a + # rename outside the docs job's allow-list is exactly the drift it exists to catch. + # The checker imports only node:fs, node:path, and node:url, so a checkout and a + # Node runtime are all it needs; the docs job runs it again inside check:source. + name: Evidence anchors + runs-on: ubuntu-24.04 + timeout-minutes: 10 + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 + with: + persist-credentials: false + submodules: false + + - name: Setup Node + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 + with: + node-version: 22.12.0 + + - name: Check documentation anchors against the source tree + run: node docs/site/scripts/check-evidence-anchors.mjs + docs: name: Docs checks needs: changes @@ -1186,6 +1210,7 @@ jobs: - release-tool - release-source-proof - evidence-tutorials + - evidence-anchors - docs - editor-extensions - client-bindings diff --git a/docs/site/AGENTS.md b/docs/site/AGENTS.md index 349f26de9..4e911fd5e 100644 --- a/docs/site/AGENTS.md +++ b/docs/site/AGENTS.md @@ -28,6 +28,77 @@ fixtures, OpenAPI, or an upstream standard. When evidence is missing, mark the claim inline with a `TODO[evidence]` MDX comment and propose a weaker claim level, rather than deleting the claim or asserting it. +`npm run check` resolves those anchors and fails when one does not. A cited +path must exist, a cited line reference must fall inside its file, and a cited +symbol must occur in at least one path the same anchor cites. Every anchor has +to cite at least one path this repository holds, since one that resolves none +has nothing to read its symbols against; pair an upstream standard with the +file that implements it rather than citing the standard alone. A citation may +start at any top-level directory the repository keeps, and `schemas/` is read +against the crate or product cited before it first and against the repository +root last, because both keep one. A path a symlink leads out of the checkout is +refused rather than read. A line reference is spelled `:12` or `:12-14` and +nothing else, so `:abc`, `:1foo`, and `:1.5` are reported rather than thrown +away; the full stop that ends a sentence on a path is punctuation and is left +alone. A citation that has drifted is a merge blocker, not a wart, so check an +anchor when you move the code it points at. Run `npm run check:evidence-anchors` alone +for the fast version. Root CI runs it twice: once inside the docs job, and once +in a job of its own that runs on every pull request, because the anchors cite +source all over the workspace and the docs job only runs for a changed path it +recognizes. A bare filename beside a cited path is read as prose when nothing +resolves, so naming a file the repo does not own, an adopter's `origins.yaml` +or a path a generated package writes, is still fine wherever the anchor +resolves some other path. A bare `.rs` sibling is the exception: only this +repository writes Rust into the stack, so a Rust filename has to resolve, and +deleting the file one names fails the check. Several files in one directory may +be cited in the compact brace form, +`crates/registry-relay-v2/src/{api,startup}.rs`, which is read as one citation +per entry, so each file it names has to exist on its own. A bare name ending in +a slash continues the directory cited before it, `deployment-projects/ then +protected-read-evidence/`, and has to exist under it. It continues that path +only where the repo holds it as a directory, so beside a file the name stays +prose: `governed/` after `package.rs`, and `output/` after the extensionless +`release/scripts/registry-release`, name directories the program writes, not +ones the repo holds. + +The check reads a symbol by its shape: `snake_case`, `SCREAMING_SNAKE_CASE`, +`UpperCamelCase`, `lowerCamelCase`, a name spelled with empty parentheses such as +`router()`, and an all-capital wire value carrying a digit such as `ES256`. +`UpperCamelCase` covers a name with an initialism run into it, `OAuthErrorCode`, +once that name carries two lower-case runs and one capital run of two or more. A +qualified name is read segment by segment: the last segment is read whatever its +shape, and each segment that qualifies it is read once it carries a shape of its +own. A dotted configuration or wire key path is read the same way, segment by +segment, once one of its segments carries a shape: +`evidence_data_request.transport_absences.credentials` is checked down to its +leaf, and a `*` standing for any key is skipped rather than looked up. + +Anything outside those shapes is prose, which leaves three gaps worth knowing. A +one-word name is not checked, because `UpperCamelCase` asks for two capitalized +chunks: the only shape that would reach `Visibility` also reaches every +sentence-initial word an anchor writes, `Evidence`, `Relay`, and `The` among +them. Spell a one-word type or variant qualified when you want it checked, +`AccessRule::Public` or `contract::Visibility`, since the last segment of a +qualified name is read whatever its shape. A qualifier is still read by shape, so +`Command::Check` puts `Check` under the check and leaves `Command` outside it. A +name nothing separates from an acronym the prose spells is not checked either: an +all-capital wire value with no digit, `EdDSA`, and a capitalized name carrying +one lower-case run, `SDMXProfile`. The only shape that reaches either also pulls +in `OpenAPI`, `SQLite`, `OpenCRVS`, and every other acronym the prose spells, +which would fire on correct anchors. A key path no segment of which carries a +shape, `sources.*.authentication.kind`, is not read at all: its segments are +among the commonest words in the tree, so a check on them would pass on any file +that happens to mention them. All three gaps are deliberate. Spell such a value +or key beside a symbol the check can see. + +Two things the check deliberately allows. Bare `path:start-end` citations still +pass: `--strict-line-refs` rejects them, but it stays off while a backlog of +them remains, and the check prints how many are left. Prefer citing a symbol +over a line range in new writing, because a symbol survives the next edit above +it. Prescriptive guidance that tells an operator to set a value is also +untouched, since the check reasons about claims describing what code does, not +about advice. + A procedure carries more than its commands: the reason for a step whose reason is not visible in the command, what an irreversible step forecloses, what failure looks like and the next move, and a `caution` or `danger` at every action that diff --git a/docs/site/package.json b/docs/site/package.json index e27eb769c..1b7448aa4 100644 --- a/docs/site/package.json +++ b/docs/site/package.json @@ -27,6 +27,7 @@ "check:docset": "node scripts/check-docset.mjs", "check:release-manifests": "python3 ../../release/scripts/registry-release validate-docsets", "check:evidence-links": "node scripts/check-evidence-links.mjs", + "check:evidence-anchors": "node scripts/check-evidence-anchors.mjs", "check:content": "node scripts/check-doc-frontmatter.mjs", "check:cli-reference": "node scripts/generate-cli-reference.mjs --check", "check:cutover": "node scripts/check-current-doc-cutover.mjs", @@ -46,7 +47,7 @@ "check:tutorial:evidence:dry-run": "bash scripts/check-evidence-tutorials.sh --dry-run", "check:links": "npm run build && npm run check:links:built", "check": "npm run check:source && npm run build && npm run check:built:current", - "check:source": "npm run check:cli-reference && npm run generate && npm run check:evidence-links && npm run check:docset && npm run check:release-manifests && npm run check:archive-lock && npm run check:content && npm run check:cutover && npm run check:notary-surface && npm run check:markdown && npm run check:style && npm run check:style:fixtures && npm run check:openapi && npm run check:config-vocabulary && npm run check:tutorial:dry-run && npm run check:tutorial:evidence:dry-run && npm run check:tutorial:discovery:dry-run && npm run check:svg", + "check:source": "npm run check:cli-reference && npm run generate && npm run check:evidence-links && npm run check:evidence-anchors && npm run check:docset && npm run check:release-manifests && npm run check:archive-lock && npm run check:content && npm run check:cutover && npm run check:notary-surface && npm run check:markdown && npm run check:style && npm run check:style:fixtures && npm run check:openapi && npm run check:config-vocabulary && npm run check:tutorial:dry-run && npm run check:tutorial:evidence:dry-run && npm run check:tutorial:discovery:dry-run && npm run check:svg", "check:built:current": "npm run check:accessibility:built && npm run check:llms:built && npm run check:seo:current && npm run check:links:current", "check:production": "npm run check:source && npm run build:dev && npm run check:production:built", "check:production:built": "DOCS_DIST_DIR=$PWD/dist/dev DOCS_PUBLIC_BASE=/dev/ npm run check:accessibility:built && DOCS_DIST_DIR=$PWD/dist/dev DOCS_PUBLIC_BASE=/dev/ npm run check:llms:built && DOCS_DIST_DIR=$PWD/dist/dev DOCS_PUBLIC_BASE=/dev/ npm run check:seo:current && DOCS_DIST_DIR=$PWD/dist/dev DOCS_PUBLIC_BASE=/dev/ npm run check:links:current", diff --git a/docs/site/scripts/check-evidence-anchors.mjs b/docs/site/scripts/check-evidence-anchors.mjs new file mode 100644 index 000000000..56456513b --- /dev/null +++ b/docs/site/scripts/check-evidence-anchors.mjs @@ -0,0 +1,740 @@ +#!/usr/bin/env node + +// Validates the {/* Evidence: ... */} anchors that carry every factual claim the +// documentation makes about the source repository: the paths they cite exist, the +// line references they carry are inside those files, and the symbols they name are +// present in at least one path the same anchor cites. + +import { readFileSync, readdirSync, realpathSync, statSync } from 'node:fs'; +import { dirname, relative, resolve, sep } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const scriptPath = fileURLToPath(import.meta.url); +const scriptDir = dirname(scriptPath); + +// Prose words that carry a symbol shape but name a language, never an item in the +// repository. Keep this list minimal: an entry here is a symbol the checker can no +// longer catch when it goes stale. +export const PROSE_SYMBOL_ALLOWLIST = new Set([ + // "the JavaScript example", "the TypeScript declarations": language names. + 'JavaScript', + 'TypeScript', +]); + +// Directories a repository-relative citation may start from: every top-level directory the +// repository keeps, because a citation into one this list omits parses as no citation at +// all and leaves its anchor checked against nothing. The entries are regular expression +// source, so a dot-directory carries its escape. +export const REPOSITORY_ROOTS = [ + 'crates', + 'products', + 'release', + 'docs', + 'docker', + 'editors', + 'external', + 'schemas', + '\\.cargo', + '\\.github', +]; +// Directories a continuation citation may start from, resolved against the crate or +// product root of the most recently cited path in the same anchor. +const CONTINUATION_ROOTS = ['src', 'tests', 'examples', 'benches', 'schemas', 'scripts']; +// The roots both lists name: a crate or product keeps a schemas/ directory of its own and +// so does the repository. A citation that starts at one is read against the unit cited +// before it first and against the repository root last, so the nearer directory wins, the +// way it does for a bare filename that may name a file kept at the root. +const SHARED_ROOTS = new Set(REPOSITORY_ROOTS.filter((root) => CONTINUATION_ROOTS.includes(root))); +// Extensions that make a bare token a sibling filename rather than ordinary prose. +const SOURCE_EXTENSIONS = [ + 'rs', + 'mjs', + 'md', + 'py', + 'sh', + 'rhai', + 'toml', + 'yaml', + 'yml', + 'jsonld', + 'json', + 'js', + 'ts', +]; +// Extensions read when a symbol has to be looked for inside a cited directory. +const TEXT_EXTENSIONS = new Set([ + ...SOURCE_EXTENSIONS, + 'txt', + 'sql', + 'snap', + 'html', + 'css', + 'lock', +]); +const SKIPPED_DIRECTORIES = new Set(['target', 'node_modules', '.git', 'dist', '.astro']); +// Where a continuation with no full path before it is read from: the site the anchor +// itself lives in, whose own src/ tree the docs pages cite. +const DOCS_SITE_ROOT = 'docs/site'; + +const ANCHOR_PATTERN = /\{\/\*\s*Evidence:([\s\S]*?)\*\/\}/g; +// A bare sibling that names a Rust source file is one the repository owns: an adopter of +// this stack writes configuration and scripts, never Rust, and a package the runtime +// generates carries none either. Every other extension a sibling may carry names a file the +// repository need not hold, so only this one turns a miss into drift. +const NAMES_RUST_SOURCE = /\.rs$/; +// A compact list of files that share a directory, `src/{api,startup}.rs`. It is read only +// where a path segment can start, so a brace group the prose itself writes, `{ claim, +// allowed }`, stays prose. +const BRACE_LIST = '\\{[A-Za-z0-9._-]+(?:,[A-Za-z0-9._-]+)+\\}'; +// What follows a citation root: path segments, any of which may be a brace list carrying the +// suffix its entries share, then an optional trailing slash and line reference. +const PATH_BODY = `(?:/(?:[A-Za-z0-9._-]+|${BRACE_LIST}[A-Za-z0-9._-]*))+/?(?::\\d+(?:-\\d+)?)?`; +const CITATION_PATTERN = new RegExp( + [ + `(?(?(?(?(?(?<=[\\s(]):\\d+(?:-\\d+)?(?![\\w-]))`, + ].join('|'), + 'g', +); +const LINE_SUFFIX = /^(?.*?)(?::(?\d+)(?:-(?\d+))?)?$/; +// What a line reference the citation pattern could not read leaves behind the token it +// follows: word characters or a hyphen, which a well-formed reference would have carried +// itself, optionally behind the colon that opens one. A colon the prose writes is followed +// by a space, and a reference the prose punctuates is followed by the punctuation, so +// neither leaves anything this reads. +const UNREAD_LINE_REFERENCE = /^:?[\w-][\w:-]*/; +// The same reference carried on past the line the pattern did read, the `.5` of `:1.5`, +// which would otherwise be thrown away and leave the citation checked against line 1. It is +// read only where the token carried a reference, so the full stop that ends a sentence on a +// path stays punctuation, and it needs a character after the dot, so a sentence ending on a +// line reference leaves nothing this reads either. +const CONTINUED_LINE_REFERENCE = /^\.[\w:.-]+/; +const WORD_PATTERN = /[A-Za-z_][A-Za-z0-9_]*(?:::[A-Za-z_][A-Za-z0-9_]*)+|[A-Za-z_][A-Za-z0-9_]*/g; +// A dotted configuration or wire key path, sources.*.authentication.kind. The word pass reads +// its segments one by one and keeps only the ones that carry a symbol shape, so this pattern is +// what puts the whole path back together before that decision is made. +const DOTTED_KEY_PATH = /(? expandBraceLists(`${before}${entry}${after}`)); +} + +// The crate, product, or top-level unit a continuation citation is resolved against. +function citationRoot(path) { + const segments = path.split('/'); + return ['crates', 'products', 'docs', 'external'].includes(segments[0]) && segments.length > 1 + ? `${segments[0]}/${segments[1]}` + : segments[0]; +} + +function joinPath(base, tail) { + return base === '' ? tail : `${base}/${tail}`; +} + +// Every citation carries the ordered candidate paths it may resolve to, and whether a +// candidate is a claim or a guess. A full repository path, a continuation anchored to +// one, and a continuation read against the docs site plainly name a repository path, so +// a miss is drift and is reported. A bare sibling filename is only a reading of the +// prose: when nothing resolves, it names a file the repository does not own (an +// adopter's configuration file, or a path inside a generated package) and is left alone. +// A bare Rust filename is the one sibling that is a claim, because only this repository +// writes Rust into the stack, so a miss there is drift like any other. +/** + * @typedef {object} Citation + * @property {string} form which reading produced it, and so which fallbacks apply + * @property {string[]} candidates the paths it may resolve to, tried in order + * @property {boolean} reportMissing whether resolving nothing is drift or prose + * @property {string} [raw] the token as the anchor spelled it + * @property {number} [start] first line of a line reference + * @property {number} [end] last line of a line reference + * @property {string} [malformedLines] the part of a line reference the pattern could not read + * @property {string} [basename] bare filename to search for when no candidate resolves + * @property {string} [searchRoot] where that search runs, the empty string for the whole tree + * @property {string} [rootCandidate] the same name as a file kept at the repository root + * @property {string} [childName] the bare name a child continues the last resolved + * directory with, which the repository has to hold as a directory for it to name one + */ + +export function parseAnchor(body, { siteRoot = DOCS_SITE_ROOT } = {}) { + /** @type {Citation[]} */ + const citations = []; + const strippedParts = []; + let cursor = 0; + let lastCitedPath; + let previous; + + for (const match of body.matchAll(CITATION_PATTERN)) { + const { full, relative: continuation, sibling, child, lines } = match.groups; + strippedParts.push(body.slice(cursor, match.index), ' '); + cursor = match.index + match[0].length; + + if (lines !== undefined) { + if (!previous) { + continue; + } + const { start, end } = splitLineReference(`_${lines}`); + citations.push({ ...previous, form: 'lines', raw: lines, start, end }); + continue; + } + + const token = full ?? continuation ?? sibling ?? child; + const { path, start, end } = splitLineReference(token); + const trimmed = path.replace(/\/$/, ''); + // A line reference the anchor spelled wrong leaves the part the pattern could not read + // outside the token: the hyphen of a range cut short in `:5-`, the word run into the + // number in `:1foo`, the whole suffix in `:abc`. Each would otherwise be thrown away, + // leaving the citation checked as the bare file or as a line the anchor never meant. + const unread = body.slice(cursor); + const malformedLines = + UNREAD_LINE_REFERENCE.exec(unread)?.[0] ?? + (start === undefined ? undefined : CONTINUED_LINE_REFERENCE.exec(unread)?.[0]); + + // A brace list stands for one citation per entry, so each file it names is resolved and + // counted on its own, and each entry reads against the path the entry before it set. + for (const cited of expandBraceLists(trimmed)) { + const parentOfLastCitedPath = + lastCitedPath === undefined || dirname(lastCitedPath) === '.' ? '' : dirname(lastCitedPath); + // A path that starts at a shared root is read as a continuation, and carries the + // repository reading of the same token as its last candidate. + const shared = full !== undefined && SHARED_ROOTS.has(cited.split('/')[0]); + /** @type {Citation} */ + let citation; + if (full !== undefined && !shared) { + citation = { form: 'full', candidates: [cited], reportMissing: true }; + } else if ((continuation !== undefined || shared) && lastCitedPath === undefined) { + citation = { + form: 'continuation', + candidates: [joinPath(siteRoot, cited), ...(shared ? [cited] : [])], + reportMissing: true, + }; + } else if (continuation !== undefined || shared) { + citation = { + form: 'continuation', + candidates: [ + joinPath(citationRoot(lastCitedPath), cited), + joinPath(parentOfLastCitedPath, cited), + ...(shared ? [cited] : []), + ], + reportMissing: true, + }; + } else if (child !== undefined) { + // A bare child names a directory inside the one cited before it, so a rename of + // that directory is drift and is reported. After a filename there is no directory + // to continue and the name is prose: `governed/` beside package.rs names a + // directory the package writes, not one the repository holds. Which of the two the + // path before it is, is a fact about the repository rather than about the token, so + // the parse records the name alone and the resolution reads the kind of the path + // the anchor reached. + if (lastCitedPath === undefined) { + continue; + } + citation = { + form: 'child', + candidates: [joinPath(lastCitedPath, cited)], + reportMissing: true, + childName: cited, + }; + } else if (lastCitedPath === undefined) { + // A bare filename that opens an anchor has no path to sit beside, so the + // repository itself is what it is read against: the file kept at the root, then a + // single unambiguous file of that name anywhere in the tree. It stays a reading of + // the prose for the same reason a sibling does, and the missing path before it is + // one more reason: nothing at all says the repository owns the name. + citation = { + form: 'sibling', + candidates: [cited], + reportMissing: NAMES_RUST_SOURCE.test(cited), + basename: cited, + searchRoot: '', + }; + } else { + citation = { + form: 'sibling', + candidates: [ + joinPath(parentOfLastCitedPath, cited), + joinPath(citationRoot(lastCitedPath), cited), + joinPath(lastCitedPath, cited), + ], + reportMissing: NAMES_RUST_SOURCE.test(cited), + basename: cited, + // A bare filename may name a file the repository keeps at its root, Cargo.toml + // or deny.toml, which sits beside no cited path at all. It is tried only after + // the search inside the cited unit, so a nearer file always wins. + rootCandidate: cited, + searchRoot: citationRoot(lastCitedPath), + }; + } + + citation.candidates = [...new Set(citation.candidates)]; + // What follows reads against the path cited last, whichever form carried it: a + // continuation moves the anchor on just as a second full path does. A sibling names + // no directory to read the next citation against, so it leaves the anchor where it + // is whether or not the repository has to hold the file it names. + if (citation.form !== 'sibling') { + lastCitedPath = citation.candidates[0]; + } + previous = citation; + citations.push({ ...citation, raw: token, start, end, malformedLines }); + } + } + + strippedParts.push(body.slice(cursor)); + return { citations, symbols: extractSymbols(strippedParts.join('')) }; +} + +// The shapes that hold a name apart from the prose around it. +function carriesSymbolShape(candidate) { + return ( + SCREAMING_SNAKE_CASE.test(candidate) || + SNAKE_CASE.test(candidate) || + UPPER_CAMEL_CASE.test(candidate) || + UPPER_CAMEL_CASE_WITH_INITIALISM.test(candidate) || + LOWER_CAMEL_CASE.test(candidate) || + UPPER_CASE_WIRE_VALUE.test(candidate) + ); +} + +export function extractSymbols(prose) { + const symbols = []; + const record = (candidate) => { + if (PROSE_SYMBOL_ALLOWLIST.has(candidate)) { + return; + } + if (!symbols.includes(candidate)) { + symbols.push(candidate); + } + }; + + // A dotted key path names one key per segment, and a segment such as the credentials of + // evidence_data_request.transport_absences.credentials carries no shape of its own, so the + // word pass below drops it and a typo there goes unreported. One segment already carrying a + // symbol shape is what holds a key path apart from a domain name, id.registrystack.org, or a + // version string, v0.9.0, whose segments name nothing to look for. + for (const match of prose.matchAll(DOTTED_KEY_PATH)) { + const segments = match[0].split('.'); + if (!segments.some(carriesSymbolShape)) { + continue; + } + for (const segment of segments) { + // A wildcard segment stands for any key rather than naming one. + if (segment !== '*') { + record(segment); + } + } + } + + for (const match of prose.matchAll(WORD_PATTERN)) { + const token = match[0]; + const segments = token.split('::'); + const candidate = segments.at(-1); + // Every segment of a qualified path names something the repository holds, so a typo + // in the module, type, or enum that qualifies the name is drift too. Segments that + // carry no symbol shape, `std` and `fs` in std::fs::read, name nothing to look for. + for (const qualifier of segments.slice(0, -1)) { + if (carriesSymbolShape(qualifier)) { + record(qualifier); + } + } + // An anchor spells a function reference with empty parentheses, so an identifier + // written that way is a name whatever its case. Parentheses that carry anything, + // "the check (see below)", are prose. + const spelledAsCall = prose.startsWith('()', match.index + token.length); + if (segments.length === 1 && !spelledAsCall && !carriesSymbolShape(candidate)) { + continue; + } + record(candidate); + } + return symbols; +} + +// The path a read would really open. A path the repository does not hold has none, and +// stays the missing citation it already was rather than becoming an escape. +function realPath(absolute) { + try { + return realpathSync(absolute); + } catch (error) { + if (error.code === 'ENOENT' || error.code === 'ENOTDIR') { + return undefined; + } + throw error; + } +} + +// A citation that climbs above the repository, by a `..` segment or by resolving outside +// the root, names nothing the documentation can cite, so it is refused before it is read. +function escapesRepository(repoRoot, path) { + if (path.split('/').includes('..')) { + return true; + } + const root = resolve(repoRoot); + const absolute = resolve(root, path); + if (absolute !== root && !absolute.startsWith(`${root}${sep}`)) { + return true; + } + // A symlink passes the check above and is then followed by both the stat and the read, + // so the real path is what decides. The root is resolved too: a macOS temporary + // directory is itself reached through a symlink, and an unresolved root would call + // every path beneath it an escape. + const realRoot = realPath(root); + const realAbsolute = realPath(absolute); + if (realRoot === undefined || realAbsolute === undefined) { + return false; + } + return realAbsolute !== realRoot && !realAbsolute.startsWith(`${realRoot}${sep}`); +} + +function entryKind(absolute) { + try { + return statSync(absolute).isDirectory() ? 'directory' : 'file'; + } catch (error) { + if (error.code === 'ENOENT' || error.code === 'ENOTDIR') { + return 'missing'; + } + throw error; + } +} + +function lineCount(text) { + const lines = text.split('\n'); + return lines.at(-1) === '' ? lines.length - 1 : lines.length; +} + +function filesUnder(directory) { + const files = []; + for (const entry of readdirSync(directory, { withFileTypes: true })) { + if (entry.isDirectory()) { + if (!SKIPPED_DIRECTORIES.has(entry.name)) { + files.push(...filesUnder(resolve(directory, entry.name))); + } + continue; + } + if (entry.isFile()) { + files.push(resolve(directory, entry.name)); + } + } + return files; +} + +function isTextFile(path) { + // A name with no extension at all is a script the repository keeps, `registry-release` or + // `justfile`, and reading it is how a symbol an anchor cites from one is found. The + // directories that hold anything else are skipped before the walk reaches them. + const name = path.split('/').at(-1); + return name.includes('.') ? TEXT_EXTENSIONS.has(name.split('.').at(-1)) : true; +} + +function wholeWordPattern(symbol) { + const escaped = symbol.replaceAll(/[.*+?^${}()|[\]\\]/g, '\\$&'); + return new RegExp(`(? { + if (!fileTexts.has(path)) { + fileTexts.set(path, readFileSync(resolve(repoRoot, path), 'utf8')); + } + return fileTexts.get(path); + }; + + const listFiles = (path) => { + if (!directoryFiles.has(path)) { + const absolute = resolve(repoRoot, path); + directoryFiles.set( + path, + entryKind(absolute) === 'directory' + ? filesUnder(absolute).map((file) => relative(repoRoot, file).replaceAll('\\', '/')) + : [], + ); + } + return directoryFiles.get(path); + }; + + // A bare sibling filename may name a file that sits elsewhere in the crate or product + // the anchor already named, so fall back to a single unambiguous match under it. The + // root is the repository itself when the filename opened the anchor and named no unit + // to search inside. + const uniqueFileNamed = (root, basename) => { + const matches = listFiles(root).filter((path) => path.endsWith(`/${basename}`)); + return matches.length === 1 ? matches[0] : undefined; + }; + + for (const page of mdxPages(contentRoot)) { + const location = relative(contentRoot, page).replaceAll('\\', '/'); + for (const anchor of extractAnchors(readFileSync(page, 'utf8'))) { + anchors += 1; + const { citations, symbols: cited } = parseAnchor(anchor.body); + const at = `${location}:${anchor.line}`; + const reportedBefore = errors.length; + const citedFiles = []; + const citedDirectories = []; + let lastResolvedFile; + // Where the anchor stands: the path the citation before this one resolved to, which + // a bare child continues. It is not the parse's first candidate, because a citation + // that resolved through a fallback left that guess naming nothing. + let lastResolvedPath; + + for (const citation of citations) { + const range = + citation.start === undefined + ? '' + : `:${citation.start}${citation.end === citation.start ? '' : `-${citation.end}`}`; + // A bare child continues a directory. Where the path before it is a file, or is + // nothing the repository holds at all, the name is prose rather than a citation: + // it names a directory that file writes at runtime, not a path the repository + // keeps, and an extensionless script is a file like any other. + let candidates = citation.candidates; + if (citation.form === 'child') { + if ( + lastResolvedPath === undefined || + entryKind(resolve(repoRoot, lastResolvedPath)) !== 'directory' + ) { + lastResolvedPath = undefined; + continue; + } + candidates = [joinPath(lastResolvedPath, citation.childName)]; + } + const escaping = candidates.find((candidate) => escapesRepository(repoRoot, candidate)); + if (escaping !== undefined) { + paths += 1; + if (range !== '') { + lineRefs += 1; + } + lastResolvedPath = undefined; + errors.push(`${at} cites ${escaping}${range}, which leaves the repository`); + continue; + } + // A cut-short suffix parses as a line the anchor never meant, so it is reported + // as the malformed reference it is rather than checked against the file. + if (citation.malformedLines !== undefined) { + paths += 1; + lineRefs += 1; + lastResolvedPath = undefined; + errors.push( + `${at} cites ${citation.raw}${citation.malformedLines}, but a line reference names a line or a first and last line`, + ); + continue; + } + let resolved = candidates.find( + (candidate) => entryKind(resolve(repoRoot, candidate)) !== 'missing', + ); + if (resolved === undefined && citation.basename !== undefined) { + resolved = uniqueFileNamed(citation.searchRoot, citation.basename); + } + // The file kept at the repository root is the last place a bare filename is looked + // for, and the one resolution the escape check above cannot have seen, because the + // root candidate is no member of the candidate list. A root file that a symlink + // leads out of the checkout is refused here rather than read, and refusing it is + // reported: it is the only thing that would have resolved. + if ( + resolved === undefined && + citation.rootCandidate !== undefined && + entryKind(resolve(repoRoot, citation.rootCandidate)) !== 'missing' + ) { + if (escapesRepository(repoRoot, citation.rootCandidate)) { + paths += 1; + if (range !== '') { + lineRefs += 1; + } + lastResolvedPath = undefined; + errors.push( + `${at} cites ${citation.rootCandidate}${range}, which leaves the repository`, + ); + continue; + } + resolved = citation.rootCandidate; + } + if (resolved === undefined && citation.form === 'lines') { + // A bare line reference that follows a filename the repository does not own + // still belongs to the last file the anchor resolved. + resolved = lastResolvedFile; + } + if (resolved === undefined && !citation.reportMissing) { + continue; + } + paths += 1; + if (range !== '') { + lineRefs += 1; + } + if (resolved === undefined) { + lastResolvedPath = undefined; + errors.push(`${at} cites ${candidates[0]}${range}, which does not exist`); + continue; + } + // A sibling names no directory to read the next citation against, so it leaves the + // anchor where it is, the same rule the parse applies to the candidate chain. + if (citation.form !== 'sibling') { + lastResolvedPath = resolved; + } + if (strictLineRefs && range !== '') { + errors.push( + `${at} cites ${resolved}${range}; line numbers drift silently, so name the symbol, test, constant, or key instead`, + ); + } + if (entryKind(resolve(repoRoot, resolved)) === 'directory') { + if (range !== '') { + errors.push(`${at} cites ${resolved}${range}, but that path is a directory`); + } + citedDirectories.push(resolved); + continue; + } + citedFiles.push(resolved); + lastResolvedFile = resolved; + if (range === '') { + continue; + } + const count = lineCount(readText(resolved)); + if (citation.start < 1 || citation.end < citation.start) { + errors.push( + `${at} cites ${resolved}${range}, but a line range starts at line 1 and ends at or after its start`, + ); + } else if (citation.start > count || citation.end > count) { + errors.push( + `${at} cites ${resolved}${range}, but the file has ${pluralLines(count)}`, + ); + } + } + + if (citedFiles.length === 0 && citedDirectories.length === 0) { + // An anchor that resolves no path has nothing to read its symbols against, so + // letting it pass would state a guarantee the check never made for it. An anchor + // whose citations were reported already fails, and naming it twice names no + // further drift. + if (errors.length === reportedBefore) { + errors.push( + citations.length === 0 + ? `${at} cites no path in this repository, so nothing it claims was checked` + : `${at} resolves none of the paths it cites, so nothing it claims was checked`, + ); + } + continue; + } + + for (const symbol of cited) { + symbols += 1; + const pattern = wholeWordPattern(symbol); + const found = + citedFiles.some((path) => pattern.test(readText(path))) || + citedDirectories.some((directory) => + listFiles(directory) + .filter((path) => isTextFile(path)) + .some((path) => pattern.test(readText(path))), + ); + if (!found) { + errors.push(`${at} cites ${symbol}, which no cited path contains`); + } + } + } + } + + return { anchors, paths, symbols, lineRefs, errors }; +} + +export function parseArguments(args) { + if (args.length === 0) { + return { strictLineRefs: false }; + } + if (args.length === 1 && args[0] === '--strict-line-refs') { + return { strictLineRefs: true }; + } + throw new Error('usage: check-evidence-anchors.mjs [--strict-line-refs]'); +} + +if (process.argv[1] && resolve(process.argv[1]) === scriptPath) { + try { + const options = parseArguments(process.argv.slice(2)); + const result = checkEvidenceAnchors(options); + const counts = + `${result.anchors} anchors, ${result.paths} cited paths, and ${result.symbols} cited symbols checked; ` + + `${result.lineRefs} line-range citations found`; + if (result.errors.length > 0) { + console.error(result.errors.join('\n')); + console.error(`Evidence anchor check failed: ${counts}.`); + process.exitCode = 1; + } else { + console.log(`Evidence anchor check passed: ${counts}.`); + } + } catch (error) { + console.error(error.message); + process.exitCode = 1; + } +} diff --git a/docs/site/scripts/check-evidence-anchors.test.mjs b/docs/site/scripts/check-evidence-anchors.test.mjs new file mode 100644 index 000000000..9c43b47c5 --- /dev/null +++ b/docs/site/scripts/check-evidence-anchors.test.mjs @@ -0,0 +1,946 @@ +import assert from 'node:assert/strict'; +import { execFileSync } from 'node:child_process'; +import { mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, resolve } from 'node:path'; +import { test } from 'node:test'; +import { fileURLToPath } from 'node:url'; + +import YAML from 'yaml'; + +import { + REPOSITORY_ROOTS, + checkEvidenceAnchors, + extractAnchors, + extractSymbols, + parseAnchor, + parseArguments, +} from './check-evidence-anchors.mjs'; + +const repositoryRoot = resolve(dirname(fileURLToPath(import.meta.url)), '../../..'); + +function write(root, path, contents) { + const target = resolve(root, path); + mkdirSync(dirname(target), { recursive: true }); + writeFileSync(target, contents); +} + +function repository(t) { + const root = mkdtempSync(resolve(tmpdir(), 'registry-evidence-anchors-')); + t.after(() => rmSync(root, { recursive: true, force: true })); + write(root, 'crates/demo/src/lib.rs', 'pub fn verify_source_shape() {}\n'); + write(root, 'crates/demo/src/other.rs', 'pub const SOURCE_LIMIT: usize = 4;\n'); + write(root, 'crates/demo/tests/cli_contract.rs', 'fn covers_the_binary() {}\n'); + write(root, 'crates/demo/tests/language_server.rs', 'fn reports_editor_diagnostics() {}\n'); + return root; +} + +function check(root, body, options = {}) { + write(root, 'docs/site/src/content/docs/page.mdx', `---\ntitle: Page\n---\n\n${body}\n`); + return checkEvidenceAnchors({ repoRoot: root, ...options }); +} + +test('extracts multi-line anchors with the line they start on', () => { + const anchors = extractAnchors('one\ntwo\n{/* Evidence: crates/demo/src/lib.rs\n holds it. */}\n'); + assert.equal(anchors.length, 1); + assert.equal(anchors[0].line, 3); + assert.match(anchors[0].body, /holds it\./); +}); + +test('accepts an anchor whose path exists and whose symbol resolves', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: crates/demo/src/lib.rs, verify_source_shape(). */}'); + assert.deepEqual(result.errors, []); + assert.equal(result.anchors, 1); + assert.equal(result.paths, 1); + assert.equal(result.symbols, 1); +}); + +test('reports a cited path that does not exist', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: crates/demo/src/absent.rs holds the check. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /^page\.mdx:5 /); + assert.match(result.errors[0], /crates\/demo\/src\/absent\.rs/); +}); + +test('reports a citation whose path climbs out of the repository', (t) => { + const root = repository(t); + const outside = resolve(root, '..', 'registry-evidence-anchors-outside.txt'); + writeFileSync(outside, 'held outside the repository\n'); + t.after(() => rmSync(outside, { force: true })); + const result = check( + root, + '{/* Evidence: crates/../../registry-evidence-anchors-outside.txt holds it. */}', + ); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/\.\.\/\.\.\/registry-evidence-anchors-outside\.txt/); + assert.match(result.errors[0], /leaves the repository/); +}); + +test('reports a citation whose real path leaves the repository through a symlink', (t) => { + const root = repository(t); + const outside = resolve(root, '..', 'registry-evidence-anchors-linked.rs'); + writeFileSync(outside, 'pub fn held_outside_the_repository() {}\n'); + t.after(() => rmSync(outside, { force: true })); + symlinkSync(outside, resolve(root, 'crates/demo/src/linked.rs')); + const result = check( + root, + '{/* Evidence: crates/demo/src/linked.rs, held_outside_the_repository(). */}', + ); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/linked\.rs/); + assert.match(result.errors[0], /leaves the repository/); +}); + +test('refuses a repository-root file a symlink leads out of', (t) => { + const root = repository(t); + const outside = resolve(root, '..', 'registry-evidence-anchors-root-linked.toml'); + writeFileSync(outside, '[bans]\nmultiple_versions = "deny"\n'); + t.after(() => rmSync(outside, { force: true })); + symlinkSync(outside, resolve(root, 'deny.toml')); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs, and deny.toml, multiple_versions. */}', + ); + assert.equal(result.errors.length, 2); + assert.match(result.errors[0], /deny\.toml/); + assert.match(result.errors[0], /leaves the repository/); + // The file outside the checkout is never read, so the symbol it holds stays unfound. + assert.match(result.errors[1], /multiple_versions/); +}); + +test('resolves a sibling beside its cited path though the root file of that name escapes', (t) => { + const root = repository(t); + const outside = resolve(root, '..', 'registry-evidence-anchors-root-shadow.rs'); + writeFileSync(outside, 'pub const SOURCE_LIMIT: usize = 9;\n'); + t.after(() => rmSync(outside, { force: true })); + symlinkSync(outside, resolve(root, 'other.rs')); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs, and other.rs, SOURCE_LIMIT. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 2); +}); + +test('reports a line reference past the end of the file with the real line count', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: crates/demo/src/lib.rs:40-42 holds it. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/lib\.rs:40-42/); + assert.match(result.errors[0], /has 1 line\b/); +}); + +test('reports a line reference that starts before the first line', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: crates/demo/src/lib.rs:0 holds it. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/lib\.rs:0/); + assert.match(result.errors[0], /starts at line 1/); +}); + +test('reports a line range that ends before it starts', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/wide.rs', 'one\ntwo\nthree\nfour\nfive\nsix\n'); + const result = check(root, '{/* Evidence: crates/demo/src/wide.rs:5-3 holds it. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/wide\.rs:5-3/); + assert.match(result.errors[0], /ends at or after its start/); +}); + +test('reports a bare line range that ends before it starts', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/wide.rs', 'one\ntwo\nthree\nfour\nfive\nsix\n'); + const result = check(root, '{/* Evidence: crates/demo/src/wide.rs:1, and :5-3. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/wide\.rs:5-3/); + assert.match(result.errors[0], /ends at or after its start/); +}); + +test('reports a line suffix the anchor cut short', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/wide.rs', 'one\ntwo\nthree\nfour\nfive\nsix\n'); + const result = check(root, '{/* Evidence: crates/demo/src/wide.rs:5- holds it. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/wide\.rs:5-/); + assert.match(result.errors[0], /a line or a first and last line/); + assert.equal(result.lineRefs, 1); +}); + +test('reports a line reference the anchor spelled with something other than a number', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: crates/demo/src/lib.rs:abc holds it. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/lib\.rs:abc/); + assert.match(result.errors[0], /a line or a first and last line/); + assert.equal(result.lineRefs, 1); +}); + +test('reports a line reference the anchor ran into the word behind it', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: crates/demo/src/lib.rs:1foo holds it. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/lib\.rs:1foo/); + assert.match(result.errors[0], /a line or a first and last line/); +}); + +test('leaves a colon the prose writes after a cited path alone', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs: it holds verify_source_shape(). */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 1); + assert.equal(result.lineRefs, 0); +}); + +test('leaves a well-formed line reference the prose punctuates alone', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/wide.rs', 'one\ntwo\nthree\nfour\nfive\nsix\n'); + const result = check( + root, + '{/* Evidence: crates/demo/src/wide.rs:1-2, crates/demo/src/wide.rs:3-4; crates/demo/src/wide.rs:5-6. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.lineRefs, 3); +}); + +test('leaves a hyphen the prose carries after a line reference alone', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/wide.rs', 'one\ntwo\nthree\nfour\nfive\nsix\n'); + const result = check(root, '{/* Evidence: crates/demo/src/wide.rs:5 - the middle of it. */}'); + assert.deepEqual(result.errors, []); +}); + +test('reports a symbol that appears in no cited path', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: crates/demo/src/lib.rs, absent_test_name. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /absent_test_name/); +}); + +test('checks a lower camel case symbol against the cited paths', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/wire.rs', 'pub struct Body { packageRevision: u32 }\n'); + const passing = check(root, '{/* Evidence: crates/demo/src/wire.rs carries packageRevision. */}'); + assert.deepEqual(passing.errors, []); + assert.equal(passing.symbols, 1); + + const failing = check(root, '{/* Evidence: crates/demo/src/wire.rs carries packageRevison. */}'); + assert.equal(failing.errors.length, 1); + assert.match(failing.errors[0], /packageRevison/); +}); + +test('checks an all-uppercase wire value against the cited paths', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/algorithms.rs', 'const ALLOWED: [&str; 2] = ["ES256", "RS256"];\n'); + const passing = check(root, '{/* Evidence: crates/demo/src/algorithms.rs allows ES256. */}'); + assert.deepEqual(passing.errors, []); + assert.equal(passing.symbols, 1); + + const failing = check(root, '{/* Evidence: crates/demo/src/algorithms.rs allows ES265. */}'); + assert.equal(failing.errors.length, 1); + assert.match(failing.errors[0], /ES265/); +}); + +test('leaves a version word and an acronym the prose spells in capitals unchecked', () => { + assert.deepEqual(extractSymbols('the V2 registry contract serves JSON over HTTP'), []); + assert.deepEqual(extractSymbols('the profile allows ES256 and RS256'), ['ES256', 'RS256']); +}); + +test('checks an upper camel case name that runs an initialism into it', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/token.rs', 'pub enum OAuthErrorCode {\n InvalidClient,\n}\n'); + const passing = check(root, '{/* Evidence: crates/demo/src/token.rs, OAuthErrorCode. */}'); + assert.deepEqual(passing.errors, []); + assert.equal(passing.symbols, 1); + + const failing = check(root, '{/* Evidence: crates/demo/src/token.rs, OAuthErrorKind. */}'); + assert.equal(failing.errors.length, 1); + assert.match(failing.errors[0], /OAuthErrorKind/); +}); + +test('leaves an acronym the prose spells with one lower-case run unchecked', () => { + assert.deepEqual(extractSymbols('the OpenAPI description of the SQLite source'), []); + assert.deepEqual(extractSymbols('the SDMX profile, the JWKS endpoint, and EdDSA'), []); + assert.deepEqual(extractSymbols('the OpenCRVS demo signs with SHA'), []); + assert.deepEqual(extractSymbols('it reads HTTPRedirectHandler'), ['HTTPRedirectHandler']); +}); + +test('accepts a symbol that appears in the second of two cited paths', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs and crates/demo/src/other.rs define SOURCE_LIMIT. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 2); +}); + +test('resolves a relative continuation against the crate root of the last full path', (t) => { + const root = repository(t); + const passing = check( + root, + '{/* Evidence: crates/demo/src/lib.rs, verify_source_shape(); src/other.rs, SOURCE_LIMIT. */}', + ); + assert.deepEqual(passing.errors, []); + assert.equal(passing.paths, 2); + + const failing = check(root, '{/* Evidence: crates/demo/src/lib.rs and src/absent.rs. */}'); + assert.equal(failing.errors.length, 1); + assert.match(failing.errors[0], /crates\/demo\/src\/absent\.rs/); +}); + +test('expands a brace list into one citation per entry', () => { + const parsed = parseAnchor('crates/demo/src/{lib,other}.rs carry it.'); + assert.deepEqual( + parsed.citations.map((citation) => [citation.form, citation.candidates[0]]), + [ + ['full', 'crates/demo/src/lib.rs'], + ['full', 'crates/demo/src/other.rs'], + ], + ); + assert.ok(parsed.citations.every((citation) => citation.reportMissing)); +}); + +test('reports the entry of a brace list that does not exist', (t) => { + const root = repository(t); + const passing = check(root, '{/* Evidence: crates/demo/src/{lib,other}.rs carry it. */}'); + assert.deepEqual(passing.errors, []); + assert.equal(passing.paths, 2); + + const failing = check(root, '{/* Evidence: crates/demo/src/{lib,absent}.rs carry it. */}'); + assert.equal(failing.errors.length, 1); + assert.match(failing.errors[0], /crates\/demo\/src\/absent\.rs/); + assert.match(failing.errors[0], /does not exist/); + assert.equal(failing.paths, 2); +}); + +test('expands a brace list a sentence ends on', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: the surfaces sit in crates/demo/src/{lib,other}.rs. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 2); +}); + +test('carries a line reference into every entry of a brace list', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: crates/demo/src/{lib,other}.rs:40 carry it. */}'); + assert.equal(result.errors.length, 2); + assert.match(result.errors[0], /crates\/demo\/src\/lib\.rs:40/); + assert.match(result.errors[1], /crates\/demo\/src\/other\.rs:40/); + assert.equal(result.lineRefs, 2); +}); + +test('expands a brace list a continuation carries', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs, then src/{other,absent}.rs. */}', + ); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/absent\.rs/); + assert.equal(result.paths, 3); +}); + +test('leaves a brace group the prose writes out of the citations', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs returns { claim, allowed }. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 1); +}); + +test('resolves a bare sibling filename against the directory of the last full path', (t) => { + const root = repository(t); + const passing = check( + root, + '{/* Evidence: crates/demo/tests/cli_contract.rs and language_server.rs pin the surfaces. */}', + ); + assert.deepEqual(passing.errors, []); + assert.equal(passing.paths, 2); + + const elsewhere = check( + root, + '{/* Evidence: crates/demo/tests/cli_contract.rs and other.rs, SOURCE_LIMIT. */}', + ); + assert.deepEqual(elsewhere.errors, []); + assert.equal(elsewhere.paths, 2); +}); + +test('resolves a bare sibling filename against the most recently cited path', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/handler.rs', 'fn prepares_the_request() {}\n'); + write(root, 'crates/demo/tests/handler.rs', 'fn covers_the_handler() {}\n'); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs; tests/cli_contract.rs; handler.rs, covers_the_handler. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 3); +}); + +test('leaves the anchor where it is when one bare sibling filename follows another', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo holds cli_contract.rs and language_server.rs. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 3); +}); + +test('resolves a bare sibling filename against the repository root', (t) => { + const root = repository(t); + write(root, 'deny.toml', '[bans]\nmultiple_versions = "deny"\n'); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs, and deny.toml, multiple_versions. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 2); +}); + +test('prefers a file inside the cited unit over the one at the repository root', (t) => { + const root = repository(t); + write(root, 'README.md', 'The workspace README names workspace_wide_only.\n'); + write(root, 'crates/demo/reference/README.md', 'The crate README names crate_local_only.\n'); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs, and README.md, crate_local_only. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 2); +}); + +test('reads a bare script filename beside the path it sits with', (t) => { + const root = repository(t); + write(root, 'crates/demo/scripts/extract.rhai', 'let extracted = source_value;\n'); + write(root, 'crates/demo/scripts/prepare.rhai', 'let request_url = source_base;\n'); + const result = check( + root, + '{/* Evidence: crates/demo/scripts/extract.rhai and prepare.rhai, request_url. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 2); +}); + +test('leaves a bare filename the repository does not own out of the path check', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs accepts the values an adopter writes in origins.yaml. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 1); +}); + +test('reports a bare sibling Rust file no cited unit holds', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/tests/cli_contract.rs and absent.rs pin the surfaces. */}', + ); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/tests\/absent\.rs, which does not exist/); + assert.equal(result.paths, 2); +}); + +test('reports a bare Rust filename that opens an anchor and names nothing', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: absent.rs, and crates/demo/src/lib.rs reads it. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /absent\.rs, which does not exist/); +}); + +test('resolves a bare child directory against the directory cited before it', (t) => { + const root = repository(t); + write(root, 'products/demo/projects/protected-read/README.md', 'It names bounded_read_shape.\n'); + const result = check( + root, + '{/* Evidence: products/demo/projects/ then protected-read/, bounded_read_shape. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 2); +}); + +test('reports a bare child directory the cited directory does not hold', (t) => { + const root = repository(t); + write(root, 'products/demo/projects/protected-read/README.md', 'It names bounded_read_shape.\n'); + const result = check(root, '{/* Evidence: products/demo/projects/ then renamed-read/. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /products\/demo\/projects\/renamed-read/); +}); + +test('reads one bare child directory against the last, so a chain resolves', (t) => { + const root = repository(t); + write(root, 'products/demo/projects/protected-read/governed/registry.yaml', 'id: demo\n'); + const result = check( + root, + '{/* Evidence: products/demo/projects/ protected-read/ governed/ holds it. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 3); +}); + +test('reports a line reference the anchor continued with a dot', (t) => { + const root = repository(t); + const continued = check(root, '{/* Evidence: crates/demo/src/lib.rs:1.5 holds it. */}'); + assert.equal(continued.errors.length, 1); + assert.match(continued.errors[0], /a line reference names a line or a first and last line/); + + // A full stop that ends the sentence is punctuation the prose wrote, not a reference the + // anchor carried on, so it leaves the line it does name alone. + const ended = check(root, '{/* Evidence: crates/demo/src/lib.rs:1. It holds the shape. */}'); + assert.deepEqual(ended.errors, []); + assert.equal(ended.lineRefs, 1); +}); + +test('reads a JavaScript or TypeScript sibling as a citation, not a dotted key path', (t) => { + const root = repository(t); + write(root, 'editors/vscode/src/extension.ts', 'export const activateEditor = 1;\n'); + write(root, 'editors/vscode/src/projectRoot.ts', 'export const rootOf = 2;\n'); + write(root, 'crates/demo/client.js', 'export const requestShape = 3;\n'); + write(root, 'crates/demo/index.js', 'export const entryShape = 4;\n'); + // Read as a dotted key path instead, the name would demand a symbol spelled `ts`. + assert.deepEqual( + parseAnchor('editors/vscode/src/extension.ts and projectRoot.ts hold it.').symbols, + [], + ); + const typescript = check( + root, + '{/* Evidence: editors/vscode/src/extension.ts and projectRoot.ts, rootOf. */}', + ); + assert.deepEqual(typescript.errors, []); + assert.equal(typescript.paths, 2); + + const javascript = check(root, '{/* Evidence: crates/demo/client.js and index.js, entryShape. */}'); + assert.deepEqual(javascript.errors, []); + assert.equal(javascript.paths, 2); +}); + +test('reads an extensionless script under a cited directory', (t) => { + const root = repository(t); + write( + root, + 'release/scripts/registry-release', + '#!/usr/bin/env python3\nartifact_inventory_errors = []\n', + ); + const result = check(root, '{/* Evidence: release/scripts/ reports artifact_inventory_errors. */}'); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 1); + assert.equal(result.symbols, 1); +}); + +test('reads a bare child against the directory the anchor resolved, not the first guess', (t) => { + const root = repository(t); + write(root, 'schemas/registry/profile/registry.schema.json', '{ "title": "profile_form" }\n'); + // The shared root resolves at the repository, the third candidate, because the crate + // cited before it keeps no schemas/ of its own. A child read against the first candidate + // instead would find no directory there and drop the citation unchecked. + const held = check( + root, + '{/* Evidence: crates/demo/src/lib.rs, then schemas/registry/ profile/, profile_form. */}', + ); + assert.deepEqual(held.errors, []); + assert.equal(held.paths, 3); + + const renamed = check( + root, + '{/* Evidence: crates/demo/src/lib.rs, then schemas/registry/ absent/. */}', + ); + assert.equal(renamed.errors.length, 1); + assert.match(renamed.errors[0], /schemas\/registry\/absent/); +}); + +test('reads a trailing-slash name that follows an extensionless file as prose', (t) => { + const root = repository(t); + write(root, 'release/scripts/registry-release', '#!/usr/bin/env python3\nprint("pack")\n'); + const result = check( + root, + '{/* Evidence: release/scripts/registry-release writes output/ and manifests/. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 1); +}); + +test('continues a bare child under an extensionless directory the repository holds', (t) => { + const root = repository(t); + write(root, 'products/demo/projects/protected-read/README.md', 'It names bounded_read_shape.\n'); + const result = check( + root, + '{/* Evidence: products/demo/projects then protected-read/, bounded_read_shape. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 2); +}); + +test('leaves a trailing-slash name that follows a cited file out of the citations', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs writes governed/ and generated/ into the package. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 1); +}); + +test('leaves a slash the prose writes out of the citations', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs decides read and/or write at https://example.org/. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 1); +}); + +test('resolves a bare filename that opens an anchor against the repository root', (t) => { + const root = repository(t); + write(root, 'deny.toml', '[bans]\nmultiple_versions = "deny"\n'); + const result = check( + root, + '{/* Evidence: deny.toml, multiple_versions, and crates/demo/src/lib.rs. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 2); +}); + +test('searches the tree for a bare filename that opens an anchor', (t) => { + const root = repository(t); + write(root, 'products/demo/reference/CONFIG.md', 'The reference names bundle_signing_key.\n'); + const result = check(root, '{/* Evidence: CONFIG.md, bundle_signing_key. */}'); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 1); +}); + +test('leaves a bare filename that opens an anchor and names nothing out of the path check', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: origins.yaml is the adopter file, crates/demo/src/lib.rs reads it. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.paths, 1); +}); + +test('resolves a bare line range against the most recently cited path', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: crates/demo/src/other.rs:1, and :305-309. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/other\.rs:305-309/); + assert.match(result.errors[0], /has 1 line\b/); +}); + +test('carries a bare line range past a filename the repository does not own', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/src/other.rs:1 writes origins.yaml, then :305-309. */}', + ); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/other\.rs:305-309/); +}); + +test('does not treat a cited filename stem as a symbol', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/tests/cli_contract.rs pins the tooling inventory. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.symbols, 0); +}); + +test('skips prose words that carry a symbol shape but are on the allowlist', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs backs the JavaScript and TypeScript bindings. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.symbols, 0); +}); + +test('reports an anchor that cites no path at all', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: the operator contract states does_not_own. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /^page\.mdx:5 /); + assert.match(result.errors[0], /cites no path in this repository/); + assert.equal(result.paths, 0); + assert.equal(result.symbols, 0); +}); + +test('reports an anchor none of whose citations resolve', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: origins.yaml carries the source_kind an adopter sets. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /resolves none of the paths it cites/); + assert.equal(result.paths, 0); + assert.equal(result.symbols, 0); +}); + +test('leaves an anchor whose citations were reported to report itself again', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: crates/demo/src/absent.rs holds does_not_own. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /crates\/demo\/src\/absent\.rs/); + assert.match(result.errors[0], /does not exist/); +}); + +test('reads a continuation with no full path before it against the docs site', (t) => { + const root = repository(t); + write(root, 'docs/site/src/data/projects.yaml', '- id: demo\n does_not_own: []\n'); + const resolved = check(root, '{/* Evidence: src/data/projects.yaml, does_not_own. */}'); + assert.deepEqual(resolved.errors, []); + assert.equal(resolved.paths, 1); +}); + +test('reports a continuation the docs site does not hold', (t) => { + const root = repository(t); + const result = check(root, '{/* Evidence: src/data/absent.yaml, does_not_own. */}'); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /docs\/site\/src\/data\/absent\.yaml/); + assert.match(result.errors[0], /does not exist/); + assert.equal(result.paths, 1); +}); + +test('counts line-range citations and fails them only under strict line references', (t) => { + const root = repository(t); + const relaxed = check(root, '{/* Evidence: crates/demo/src/lib.rs:1, verify_source_shape(). */}'); + assert.deepEqual(relaxed.errors, []); + assert.equal(relaxed.lineRefs, 1); + + const strict = check( + root, + '{/* Evidence: crates/demo/src/lib.rs:1, verify_source_shape(). */}', + { strictLineRefs: true }, + ); + assert.equal(strict.errors.length, 1); + assert.match(strict.errors[0], /crates\/demo\/src\/lib\.rs:1/); + assert.match(strict.errors[0], /line numbers drift/); +}); + +test('takes the last segment of a qualified symbol path', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/codes.rs', 'pub enum ProblemCode { AuditUnavailable }\n'); + const result = check( + root, + '{/* Evidence: crates/demo/src/codes.rs, ProblemCode::AuditUnavailable. */}', + ); + assert.deepEqual(result.errors, []); +}); + +test('checks every segment of a qualified symbol path that carries a symbol shape', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/codes.rs', 'pub enum ProblemCode { AuditUnavailable }\n'); + const result = check( + root, + '{/* Evidence: crates/demo/src/codes.rs, ProblmCode::AuditUnavailable. */}', + ); + assert.equal(result.errors.length, 1); + assert.match(result.errors[0], /ProblmCode/); + assert.equal(result.symbols, 2); +}); + +test('leaves the segments of a qualified path that name no symbol unchecked', () => { + assert.deepEqual(extractSymbols('std::fs::read_to_string reads it.'), ['read_to_string']); + assert.deepEqual(extractSymbols('ProblemCode::AuditUnavailable is returned.'), [ + 'ProblemCode', + 'AuditUnavailable', + ]); +}); + +test('checks every segment of a dotted key path one segment gives a shape', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/keys.rs', 'const KEY: &str = "transport_absences.credentials";\n'); + const passing = check( + root, + '{/* Evidence: crates/demo/src/keys.rs holds transport_absences.credentials. */}', + ); + assert.deepEqual(passing.errors, []); + assert.equal(passing.symbols, 2); + + const failing = check( + root, + '{/* Evidence: crates/demo/src/keys.rs holds transport_absences.credntials. */}', + ); + assert.equal(failing.errors.length, 1); + assert.match(failing.errors[0], /credntials/); +}); + +test('leaves a dotted token no segment gives a shape out of the symbols', (t) => { + const root = repository(t); + const result = check( + root, + '{/* Evidence: crates/demo/src/lib.rs is served from id.registrystack.org. */}', + ); + assert.deepEqual(result.errors, []); + assert.equal(result.symbols, 0); +}); + +test('reads a dotted key path segment by segment and a domain name not at all', () => { + assert.deepEqual(extractSymbols('the request sets transport_absences.credentials'), [ + 'transport_absences', + 'credentials', + ]); + assert.deepEqual(extractSymbols('published at id.registrystack.org since v0.9.0'), []); +}); + +test('skips the wildcard segment of a dotted key path', () => { + assert.deepEqual(extractSymbols('sources.*.authentication.source_kind names it'), [ + 'sources', + 'authentication', + 'source_kind', + ]); +}); + +test('reads an identifier spelled with empty parentheses as a symbol', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/app.rs', 'pub fn router() -> Router {}\n'); + const passing = check(root, '{/* Evidence: crates/demo/src/app.rs builds router(). */}'); + assert.deepEqual(passing.errors, []); + assert.equal(passing.symbols, 1); + + const failing = check(root, '{/* Evidence: crates/demo/src/app.rs builds routes(). */}'); + assert.equal(failing.errors.length, 1); + assert.match(failing.errors[0], /routes/); +}); + +test('leaves a word the prose follows with a parenthesis out of the symbols', () => { + assert.deepEqual(extractSymbols('the check (see below) and the note(s) it carries'), []); + assert.deepEqual(extractSymbols('router(), prepare()'), ['router', 'prepare']); +}); + +test('keeps ordinary prose words out of the symbols a lower camel case name is read from', () => { + assert.deepEqual(extractSymbols('the source an evidence deployment reads is packageRevision.'), [ + 'packageRevision', + ]); +}); + +test('parses the strict line reference flag', () => { + assert.deepEqual(parseArguments([]), { strictLineRefs: false }); + assert.deepEqual(parseArguments(['--strict-line-refs']), { strictLineRefs: true }); + assert.throws(() => parseArguments(['--unknown']), /usage: check-evidence-anchors\.mjs/); +}); + +test('parses citations and symbols without touching the filesystem', () => { + const parsed = parseAnchor( + 'crates/demo/src/lib.rs:12-14 defines verify_source_shape() and SOURCE_LIMIT.', + ); + assert.deepEqual( + parsed.citations.map((citation) => citation.candidates[0]), + ['crates/demo/src/lib.rs'], + ); + assert.deepEqual(parsed.citations[0].start, 12); + assert.deepEqual(parsed.citations[0].end, 14); + assert.deepEqual(parsed.symbols, ['verify_source_shape', 'SOURCE_LIMIT']); +}); + +test('root CI runs the anchor check on every pull request and gates the branch on it', () => { + const workflow = YAML.parse( + readFileSync(resolve(repositoryRoot, '.github/workflows/ci.yml'), 'utf8'), + ); + const command = 'node docs/site/scripts/check-evidence-anchors.mjs'; + const running = Object.entries(workflow.jobs).filter(([, job]) => + (job.steps ?? []).some((step) => (step.run ?? '').includes(command)), + ); + assert.equal(running.length, 1); + const [jobId, job] = running[0]; + + // The anchors cite source across the whole workspace, so a job the changed-path + // classifier can skip is a gate that misses the renames it exists to catch. + assert.equal(job.if, undefined); + assert.deepEqual(job.needs ?? [], []); + // No install and no build: the checker imports only node:fs, node:path, and node:url. + assert.equal( + (job.steps ?? []).some((step) => (step.run ?? '').includes('npm ci')), + false, + ); + // A job the aggregate does not wait on can fail without blocking the branch. + assert.ok(workflow.jobs['ci-result'].needs.includes(jobId)); +}); + +test('names every top-level directory the repository tracks as a citation root', () => { + // A citation root the list does not name parses as no citation at all, so the anchor + // carrying it is checked against nothing. Git is what says which directories the + // repository keeps: a listing of the checkout also carries build output and local + // tooling, and which of those are present differs between a clean CI checkout and a + // working machine, so a listing would fail for reasons that are not drift. + const tracked = execFileSync('git', ['ls-tree', '-d', '--name-only', 'HEAD'], { + cwd: repositoryRoot, + encoding: 'utf8', + }) + .split('\n') + .filter((name) => name !== ''); + assert.ok(tracked.length > 0); + // The roots are regular expression source, so a dot-directory carries its escape. + const named = new Set(REPOSITORY_ROOTS.map((root) => root.replaceAll('\\', ''))); + assert.deepEqual( + tracked.filter((name) => !named.has(name)), + [], + ); +}); + +test('reads a citation into a top-level directory beside the crates and products', (t) => { + const root = repository(t); + write(root, 'editors/vscode/package.json', '{ "contributes": { "packageRevision": 1 } }\n'); + write(root, 'docker/compose/docker-compose.yaml', 'services:\n evidence: {}\n'); + write(root, '.cargo/config.toml', '[build]\nrustflags = ["--cfg", "source_neutral"]\n'); + const passing = check( + root, + '{/* Evidence: editors/vscode/package.json, packageRevision; docker/compose/docker-compose.yaml; .cargo/config.toml, source_neutral. */}', + ); + assert.deepEqual(passing.errors, []); + assert.equal(passing.paths, 3); + + const failing = check(root, '{/* Evidence: editors/vscode/absent.json carries it. */}'); + assert.equal(failing.errors.length, 1); + assert.match(failing.errors[0], /editors\/vscode\/absent\.json/); + assert.match(failing.errors[0], /does not exist/); +}); + +test('reads a citation that starts at a shared root nearest first', (t) => { + const root = repository(t); + write(root, 'crates/demo/schemas/authoring/registry.schema.json', '{ "title": "authoring_form" }\n'); + write(root, 'schemas/registry-notary.config.schema.json', '{ "title": "notary_config" }\n'); + const nearer = check( + root, + '{/* Evidence: crates/demo/src/lib.rs, then schemas/authoring/registry.schema.json, authoring_form. */}', + ); + assert.deepEqual(nearer.errors, []); + assert.equal(nearer.paths, 2); + + const atTheRoot = check( + root, + '{/* Evidence: schemas/registry-notary.config.schema.json, notary_config. */}', + ); + assert.deepEqual(atTheRoot.errors, []); + assert.equal(atTheRoot.paths, 1); +}); + +test('reads a one-word name only where the anchor spells it qualified', (t) => { + const root = repository(t); + write(root, 'crates/demo/src/rule.rs', 'pub enum AccessRule {\n Public(String),\n}\n'); + // A one-word name carries no shape holding it apart from a capitalized prose word, + // and every sentence an anchor opens starts with one. + assert.deepEqual(extractSymbols('AccessRule is Public or Protected'), ['AccessRule']); + // The last segment of a qualified name is read whatever its shape; a qualifier is + // read by shape, so a one-word type that only ever qualifies stays outside the check. + assert.deepEqual(extractSymbols('AccessRule::Public reads it'), ['AccessRule', 'Public']); + assert.deepEqual(extractSymbols('Command::run() reads it'), ['run']); + + const bare = check(root, '{/* Evidence: crates/demo/src/rule.rs, AccessRule is Protectd. */}'); + assert.deepEqual(bare.errors, []); + const qualified = check(root, '{/* Evidence: crates/demo/src/rule.rs, AccessRule::Protectd. */}'); + assert.equal(qualified.errors.length, 1); + assert.match(qualified.errors[0], /Protectd/); +}); diff --git a/docs/site/src/content/docs/explanation/discovery-as-an-index.mdx b/docs/site/src/content/docs/explanation/discovery-as-an-index.mdx index d5531e7a0..0e9b6216c 100644 --- a/docs/site/src/content/docs/explanation/discovery-as-an-index.mdx +++ b/docs/site/src/content/docs/explanation/discovery-as-an-index.mdx @@ -94,8 +94,8 @@ every required type. For Registry Relay, the application searches the public sem operation family as one correlated tuple, then explicitly selects one record. Discovery preserves the complete Evidence resolution context, Relay tuple, catalog revision, and origin provenance. -{/* Evidence: `crates/registry-discovery/src/query.rs`, `Directory::resolve_evidence_types()` and - `Directory::search_services()`; `crates/registry-discovery-client/src/selection.rs`, +{/* Evidence: `crates/registry-discovery/src/query.rs`, the `query::Directory` type with + `Directory::resolve_evidence_types()` and `Directory::search_services()`; `crates/registry-discovery-client/src/selection.rs`, `EvidenceTypeResolveSelectionExt`, `ServiceSearchSelectionExt::select_evidence()`, `ServiceSearchSelectionExt::select_relay()`, and the typed selection fields. */} diff --git a/docs/site/src/content/docs/explanation/dpi-safeguards-alignment.mdx b/docs/site/src/content/docs/explanation/dpi-safeguards-alignment.mdx index b8e0dd7d6..a8e59b3c6 100644 --- a/docs/site/src/content/docs/explanation/dpi-safeguards-alignment.mdx +++ b/docs/site/src/content/docs/explanation/dpi-safeguards-alignment.mdx @@ -8,7 +8,7 @@ source_repos: - registry-relay - registry-manifest - registry-evidence -last_reviewed: "2026-08-11" +last_reviewed: "2026-08-22" doc_type: explanation locale: en standards_referenced: @@ -90,10 +90,10 @@ framework. | Can other systems interoperate? | Manifest emits standards-shaped metadata; Evidence Gateway publishes a product-level OpenAPI document, and Relay serves an OpenAPI description generated from the deployment's own compiled contract at `GET /openapi.json`; Evidence Gateway can serialize one assertion as an SD-JWT VC under a frozen local profile. | These are scoped adoption claims, not blanket conformance to every named standard. Relay's OpenAPI describes one deployment, so two deployments do not share an API document. The SD-JWT VC profile is a second encoding of one response, and it excludes OpenID for Verifiable Credential Issuance (OID4VCI) in every part. | {/* Evidence: PurposeConstraint { claim, allowed } and AuthorityRowBinding over a token claim or the - token principal, crates/registry-relay-v2/src/contract.rs:966-976, enforced in - RelayAuthenticator::authorize(), crates/registry-relay-v2/src/auth.rs:204-257; the closed - 26-variant ProblemCode set and its PROBLEM_BASE constant, - crates/registry-relay-v2/src/problem.rs:14-45; the two fixed transforms are partial-string and + token principal, crates/registry-relay-v2/src/contract.rs, enforced in + RelayAuthenticator::authorize(), crates/registry-relay-v2/src/auth.rs; the closed 26-entry + ProblemCode catalog, where every entry carries its own literal type_uri and Relay has no + shared base constant, crates/registry-relay-http-contract/src/lib.rs; the two fixed transforms are partial-string and date-precision with year and year-month only; every audit failure maps to ProblemCode::AuditUnavailable (503) with no configuration to disable it, crates/registry-relay-v2/src/api.rs; the OpenAPI document is generated from the compiled diff --git a/docs/site/src/content/docs/explanation/integration-patterns.mdx b/docs/site/src/content/docs/explanation/integration-patterns.mdx index dd7158daf..b937e43ae 100644 --- a/docs/site/src/content/docs/explanation/integration-patterns.mdx +++ b/docs/site/src/content/docs/explanation/integration-patterns.mdx @@ -154,7 +154,8 @@ Deployment-local facts, including where the database file actually is, stay in ` never enter the reviewed contract, so the same contract can be reviewed once and deployed in staging and production without editing it. -{/* Evidence: the SourceProfile enum declares the two variants Snapshot and LiveReadOnly, and the +{/* Evidence: the SourceProfile enum declares the two variants SourceProfile::Snapshot and + SourceProfile::LiveReadOnly, and the RelayRuntime struct carries sources, package path, listener, issuer, audit sink, limits, and quotas, and nothing that can add or widen a resource, operation, access profile, or disclosure decision, both in crates/registry-relay-v2/src/contract.rs. */} @@ -239,7 +240,8 @@ If you are reading an integration guide that describes source scripts, same-orig API-key placement for Relay, it describes the retired runtime. See [Known limitations](../known-limitations/) for the full list of what went with it. -{/* Evidence: the SourceProfile enum declares exactly Snapshot and LiveReadOnly, and the +{/* Evidence: the SourceProfile enum declares exactly SourceProfile::Snapshot and + SourceProfile::LiveReadOnly, and the RelayRuntime struct's RuntimeSource accepts a source only as a path, both in crates/registry-relay-v2/src/contract.rs; the crate holds no HTTP source client. Read-only opening is enforced in crates/registry-platform-sqlite (SQLITE_OPEN_READ_ONLY | diff --git a/docs/site/src/content/docs/explanation/records-stay-home.mdx b/docs/site/src/content/docs/explanation/records-stay-home.mdx index 1a3ee681c..dbfef5972 100644 --- a/docs/site/src/content/docs/explanation/records-stay-home.mdx +++ b/docs/site/src/content/docs/explanation/records-stay-home.mdx @@ -7,7 +7,7 @@ source_repos: - registry-stack - registry-evidence - registry-relay -last_reviewed: "2026-08-21" +last_reviewed: "2026-08-22" doc_type: explanation locale: en standards_referenced: @@ -170,9 +170,11 @@ the caller's own authority through a bound `:row_authority` parameter derived fr or from the token's principal identifier. Two fixed transforms can narrow a property further: a partial string reveal (`***`) and a date reduced to year or year-month. Relay signs nothing; its responses carry no assertion. -{/* Evidence: AccessRule is Public | Protected{scope, purpose, authorityRowBinding} and - AuthorityRowBinding has a claim variant and a principal variant, - crates/registry-relay-v2/src/contract.rs; row authority is injected as a bound +{/* Evidence: an access profile is AccessRule::Public or AccessRule::Protected, whose + ProtectedAccess carries scope, purpose, and authorityRowBinding, and a row binding is + AuthorityRowBinding::Claim or AuthorityRowBinding::Principal, + crates/registry-relay-v2/src/contract.rs; the authorityRowBinding key is spelled that way in + crates/registry-relayctl/schemas/authoring/registry.schema.json; row authority is injected as a bound parameter with an exact-equality COLLATE BINARY predicate, never string concatenation, crates/registry-relay-v2/src/sqlite_runtime.rs add_row_authority(). */} Evidence Gateway returns the values a requirement declares rather than the source row; keeping that diff --git a/docs/site/src/content/docs/explanation/threat-model.mdx b/docs/site/src/content/docs/explanation/threat-model.mdx index 4f3db6785..5f6259dd7 100644 --- a/docs/site/src/content/docs/explanation/threat-model.mdx +++ b/docs/site/src/content/docs/explanation/threat-model.mdx @@ -135,8 +135,9 @@ static-credential mode and no API-key mode; the runtime reads no credential from {/* Evidence: AuthenticationRuntime carries only an optional issuer, crates/registry-relay-v2/src/contract.rs; validate_runtime_contract requires an issuer when the contract has protected access, crates/registry-relay-v2/src/startup.rs; - RelayAuthenticator::authorize() admits Public unconditionally and requires a scoped - principal for Protected, crates/registry-relay-v2/src/auth.rs. */} + RelayAuthenticator::authorize() admits CompiledAccess::Public unconditionally and + requires a scoped principal for CompiledAccess::Protected, + crates/registry-relay-v2/src/auth.rs. */} Evidence Gateway runs one reviewed OIDC bearer profile with exactly one trusted issuer and exact audience, token type, and algorithm allowlists, and one configured principal claim with no `client_id`, `azp`, header, or request fallback: missing data denies rather than diff --git a/docs/site/src/content/docs/explanation/trusted-context-constraints.mdx b/docs/site/src/content/docs/explanation/trusted-context-constraints.mdx index ca10ba209..7a6841057 100644 --- a/docs/site/src/content/docs/explanation/trusted-context-constraints.mdx +++ b/docs/site/src/content/docs/explanation/trusted-context-constraints.mdx @@ -54,7 +54,8 @@ is set by `metadataVisibility.processing`, which is public, operation-bound, or {/* Evidence: ProcessingDescription { id, operationRefs, purpose, recipientClass, legalBasisRef, dpvProfileRef, safeguards } with deny_unknown_fields, crates/registry-relay-v2/src/contract.rs; MetadataVisibility carries a processing - field over the closed Visibility enum Public | OperationBound | OperatorOnly, + field over the closed contract::Visibility enum, whose variants are Visibility::Public, + Visibility::OperationBound, and Visibility::OperatorOnly, crates/registry-relay-v2/src/contract.rs; the compiler requires every legalBasisRef and dpvProfileRef to resolve to a governed file it seals, crates/registry-relay-v2/src/compiler.rs. */} diff --git a/docs/site/src/content/docs/map/boundaries-and-map.mdx b/docs/site/src/content/docs/map/boundaries-and-map.mdx index fe412841a..811e2c77f 100644 --- a/docs/site/src/content/docs/map/boundaries-and-map.mdx +++ b/docs/site/src/content/docs/map/boundaries-and-map.mdx @@ -108,8 +108,8 @@ Registry Manifest is a pure library and CLI with no runtime data dependencies. - Production source configuration. Source file paths, source view and column names, scopes, and other deployment details belong in the Registry Relay registry contract and runtime file, not in a portable manifest. Relay does not read a manifest to find them. - {/* Evidence: contracts.yaml, registry-manifest.metadata-yaml consumer note; the V2 registry - contract and runtime schemas carry no manifest reference, + {/* Evidence: docs/site/src/data/contracts.yaml, registry-manifest.metadata-yaml consumer + note; the V2 registry contract and runtime schemas carry no manifest reference, crates/registry-relay-v2/src/contract.rs. */} - Evidence Gateway deployment configuration. An Evidence Gateway process loads one immutable governed bundle and one closed runtime file mounted read-only at startup, and reads no portable manifest. @@ -142,8 +142,8 @@ It does not own: speaks no source protocol, holds no source credential, and makes no outbound call to a source, so a CSV, spreadsheet, Parquet, PostgreSQL, or HTTP registry has to be turned into SQLite by something else before Relay can serve it. - {/* Evidence: the SourceProfile enum has exactly two variants, Snapshot and LiveReadOnly, - crates/registry-relay-v2/src/contract.rs. */} + {/* Evidence: the SourceProfile enum has exactly two variants, SourceProfile::Snapshot and + SourceProfile::LiveReadOnly, crates/registry-relay-v2/src/contract.rs. */} - Portable metadata schema ownership. The `metadata.yaml` manifest format and its renderers are owned by Registry Manifest. Relay does not consume a manifest and does not emit one: its artifact generator produces no `registry-manifest.yaml`, and a test holds that. diff --git a/docs/site/src/content/docs/security/evidence.mdx b/docs/site/src/content/docs/security/evidence.mdx index 958cfe2de..42fb618df 100644 --- a/docs/site/src/content/docs/security/evidence.mdx +++ b/docs/site/src/content/docs/security/evidence.mdx @@ -5,7 +5,7 @@ status: current owner: registry-docs source_repos: - registry-stack -last_reviewed: "2026-08-07" +last_reviewed: "2026-08-22" doc_type: explanation locale: en standards_referenced: [] @@ -223,7 +223,7 @@ does not otherwise touch. Owner-only key files. Each secret file below the configured `secretProviders.file.root` must be a regular, non-symlink file owned by the -service identity with mode `0600`; the file provider rejects anything else. +service identity with mode `0400` or `0600`; the file provider rejects anything else. Audit and subject-binding secret files must contain independently generated raw key bytes and be at least 32 bytes. The runtime derives separated audit-chain and identifier subkeys from each master, but Evidence Gateway also requires the two secret references and resolved diff --git a/docs/site/src/content/docs/security/hardening-checklist.mdx b/docs/site/src/content/docs/security/hardening-checklist.mdx index 29288190c..69f16080c 100644 --- a/docs/site/src/content/docs/security/hardening-checklist.mdx +++ b/docs/site/src/content/docs/security/hardening-checklist.mdx @@ -8,7 +8,7 @@ source_repos: - registry-evidence - registry-mint - registry-platform -last_reviewed: "2026-08-11" +last_reviewed: "2026-08-22" doc_type: how-to locale: en standards_referenced: [] @@ -80,7 +80,8 @@ Relay package. It assumes you have already configured your services per filename with no directory component, so a nested or traversing path is rejected rather than resolved. No secret may exceed 64 KiB. - Relay: on Unix, a `secret:file/` target must be a regular file owned by the running user - with mode exactly `0600` and a link count of one; a hard-linked file is refused under every name. + with mode `0400` or `0600` and a link count of one; a hard-linked file is refused under every + name. Relay refuses to start at all on a non-Unix target, because the runtime file's own path-trust check fails closed where Unix ownership and sticky-directory semantics are unavailable, so do not treat a non-Unix host as a degraded-but-working option. @@ -309,7 +310,7 @@ serve, and a startup that fails rather than degrades. protected_contracts_require_issuer_lists_require_cursor_and_lookups_require_quota, crates/registry-relay-v2/src/startup.rs:469-492; QuotaLimiter is an in-memory per-operation token bucket, crates/registry-relay-v2/src/server.rs:359-401; MetadataVisibility over the - closed Visibility enum, crates/registry-relay-v2/src/contract.rs:1042-1060. */} + closed contract::Visibility enum, crates/registry-relay-v2/src/contract.rs:1042-1060. */} - Evidence Gateway serves one operator-controlled trust domain per process. Mutually distrustful issuers or customers, or one issuer whose clients carry the same authority under different claim names, diff --git a/docs/site/src/content/docs/security/index.mdx b/docs/site/src/content/docs/security/index.mdx index ed3530fdc..1cddb6379 100644 --- a/docs/site/src/content/docs/security/index.mdx +++ b/docs/site/src/content/docs/security/index.mdx @@ -9,7 +9,7 @@ source_repos: - registry-platform - registry-evidence - registry-mint -last_reviewed: "2026-08-11" +last_reviewed: "2026-08-22" doc_type: explanation locale: en standards_referenced: [] @@ -184,8 +184,8 @@ process does not serve. Those checks are the security surface worth reviewing: issuer, and a contract that declares any lookup requires quotas. Neither is a warning. - Secrets are referenced, never inlined. The runtime file accepts exactly two reference grammars, `secret:env/` and `secret:file/`, and nothing else. A file secret must be a single - flat filename, owned by the running user, with mode exactly `0600` and a link count of one, and - no secret may exceed 64 KiB. + flat filename, owned by the running user, with mode `0400` or `0600` and a link count of one, + and no secret may exceed 64 KiB. - The source is opened read-only and pinned. Relay opens SQLite with `SQLITE_OPEN_READ_ONLY | SQLITE_OPEN_URI | SQLITE_OPEN_NO_MUTEX`, refuses a symlink, refuses a `-wal` or `-journal` sidecar beside a snapshot, and requires a snapshot to sit on a read-only @@ -204,9 +204,9 @@ process does not serve. Those checks are the security surface worth reviewing: lookups, and #[cfg(not(unix))] validate_runtime_path returns StartupError::RuntimeInvalid, crates/registry-relay-v2/src/startup.rs:96-104 and :469-492; RelayRuntime is a closed deny_unknown_fields schema, crates/registry-relay-v2/src/contract.rs:1062-1079; the two secret - grammars, the 64 KiB bound, and the uid/0600/nlink==1 file checks with their tests - references_use_only_the_two_exact_contract_grammars and - file_secret_uses_open_file_owner_and_exact_mode_checks, + grammars, the 64 KiB bound, and the uid, 0400-or-0600 mode, and nlink==1 file checks with + their tests references_use_only_the_two_exact_contract_grammars and + file_secret_accepts_only_owner_read_and_optional_owner_write_modes, crates/registry-platform-config/src/secrets.rs; the open flags, crates/registry-platform-sqlite/src/schema.rs:131-133 and src/statement.rs:494-496; sidecar and symlink refusal and the read-only-or-non-writable requirement, diff --git a/docs/site/src/content/docs/verify/index.mdx b/docs/site/src/content/docs/verify/index.mdx index 1756661d1..4716964af 100644 --- a/docs/site/src/content/docs/verify/index.mdx +++ b/docs/site/src/content/docs/verify/index.mdx @@ -5,7 +5,7 @@ status: current owner: registry-docs source_repos: - registry-stack -last_reviewed: "2026-08-11" +last_reviewed: "2026-08-22" doc_type: how-to locale: en standards_referenced: [] @@ -64,7 +64,8 @@ records across formats or the same entity tag, passes only in a full run. {/* Evidence: crates/registry-relay-v2/src/fixtures.rs compile_fixture_plan() skips every step whose id differs from the selection; assert_expectations() fails recordsEquivalentTo and etagSameAs when the referenced observation is - absent. */} + absent; both keys are spelled that way in + products/relay-v2/acceptance/business-registry/expected-http.yaml. */} ## Generate the review inputs