fix(assembly): ask the linter's own list whether a page's lang is a language

[bbertucc]

Closes #196.

preferredTag asked two questions — is this shaped like a tag, and does the registry have a preferred form for it. html-lang-valid asks a third: is it a language at all. Canonicalization is a syntax check plus the registry's alias table, so a subtag with a preferred form gets repaired (kor → ko, which is what it was added for) while one in no table has nothing to look up and passed through untouched onto the root.

So the country code written where the language code belongs put a serious violation on the one element this project writes — the exact regression the shape check exists to prevent, through the part of the question shape cannot answer.

What changed

The question now goes to axe.utils.validLangs() — the IANA registry range html-lang-valid itself validates against (8,268 primary subtags, shipped as a trie), read out of the same pinned axe-core the gate runs. Agreement is by construction, not by measurement. That matters here specifically: the guard had been widened three times (shape check → alias table → NOT_AN_ANSWER), each pass closing the instances demonstrated to it without asking what the linter asks, which is how this shipped a third time.

It lives in lint.ts (which already owns the axe dependency) as isKnownLanguage, and assembly.ts asks it about the primary subtag of the canonical value — not the whole tag, since ko-KOREAN is axe-clean and is not a whole tag any list holds.

Why not Intl.DisplayNames, which the issue proposed

The proposal was well measured and I ran it first: 87 tags, 0 root violations, so it does close #196. Then I measured what it costs, which the issue's 60-tag set didn't reach — CLDR display data has no English name for the obscure end of ISO 639-3 that the registry lists:

of 66 obscure ISO 639-3 codes: both accept 56, we refuse what axe accepts 10, axe refuses 0
aaa, aab, abt, aby, acd, adz, ahr, ajz, lns, ttj  →  axe accepts, CLDR has no name

Refusal means the root keeps en — the safe direction, as the issue says, but a document in a small language being announced as English is this amendment's own defect, reintroduced for the readers least able to absorb it. Under the axe list those ten keep their root label.

CLDR stays as the fallback, because axe.utils is a runtime export axe's own .d.ts does not declare and its semver does not cover. Read once behind a guard (length-checked too: a truncated list would refuse every language on earth and read as the guard working), and test/document-lang.test.ts asserts languageListSource === "axe", so an axe bump that drops it fails the suite instead of quietly narrowing who gets a labelled document.

Measured, after

Same body each time, runAxe on the assembled shell:

written by the page	root	root violation
cn jp cz dk gr ua vn il	en	none
xxy zzz qqq abcd	en	none
kor spa fra deu eng zho iw in ji	ko es fr de en zh he id yi	none
sh art-lojban tl	sr-Latn jbo fil	none
aaa aab abt lns ttj (CLDR-unnamed)	as written	none
haw chr fil yue ceb tlh nqo grc jbo qu	as written	none
und zxx mul mis qaa qtz	en	none
ko_KR ko-x Korean x-klingon ""	en	none

61 tags, 38 derive a root language, 0 violations on the root. NOT_AN_ANSWER stays — the registry lists und, zxx, mul, mis and qaa–qtz, which is exactly why they're refused here.

The trap this does not close

kr is Kanuri as well as the country code for Korea; se is Northern Sami as well as Sweden; no is Norwegian. A Korean page writing lang="kr" gets a Kanuri root, and no membership test can know that — the value is a language, the linter accepts it, and refusing it would refuse the derivation to every real Kanuri document. What corrects it is the review loop rewriting the fragment's own lang, or nothing. Recorded in the test and in PRD §7.4 rather than left for the next measurement to rediscover.

Tests

test/document-lang.test.ts's every language the shell will declare is one the linter accepts goes from 11 rows to 34, each a real runAxe call against the assembled shell (445 ms total) — the #196 class, the repairs, the CLDR-unnamed languages, the registry placeholders and the malformed values. The derivation table gains the refusals and the two "this is genuinely a language" rows. Plus the new the language list is the linter's own, not the fallback.

npx tsc --noEmit clean · npm test 776 pass / 0 fail · bash test/e2e.sh ALL ENDPOINTS PASSED ✅

🤖 Generated with Claude Code

[claude]

All six checks pass. I verified the load-bearing claim rather than taking the PR body for it, and it holds:

• html-lang-valid → check valid-lang evaluates isValidLang(getBaseLang(langVal)) (node_modules/axe-core/axe.js:31901-31913, :25480-25482), getBaseLang is lang.trim().split("-")[0].toLowerCase() (:18542), and isValidLang walks the same trie axe.utils.validLangs() reads (:7089, :20162). So isKnownLanguage(canonical.split("-")[0] ?? "") at src/pipeline/assembly.ts:138 is the gate's own predicate applied to the gate's own argument — the "agreement by construction" claim is exact, including the primary-subtag-only scope. validLangs() returned 8268 lowercase codes here, with kor/cn/jp absent and iw/in/und/qaa present, which is what the comments say.
• Swept all 8268 accepted subtags through LANG_TAG + NOT_AN_ANSWER + Intl.getCanonicalLocales: 0 have a canonical base subtag outside the list, so the new refusal costs no real language its root label. Same sweep for values that canonicalize into a placeholder — which would slip past NOT_AN_ANSWER (applied to the raw value) and then pass the membership test, since und/zxx/mul/qaa are all in the list — also 0 reachable, so the check ordering is safe.
• preferredTag returns either canonical or a value case-equal to it, so the tag that lands on the root always has the base subtag that was checked. test/document-lang.test.ts is covered by node --test "test/*.test.ts"; no registration gap.

Non-blocking notes

1. src/pipeline/lint.ts:249 builds the documented fallback eagerly at module scope:

   const LANGUAGE_NAMES = new Intl.DisplayNames(["en"], { type: "language", fallback: "none" });

   On the primary path that object is never touched, and on a runtime without Intl.DisplayNames (Node built --without-intl) the TypeError is raised while importing lint.ts — so the failure mode is "the pipeline module will not load" rather than "the derivation degrades to CLDR", which is the opposite of what the guard around AXE_LANGUAGES buys. Latent: official Node 24 ships full ICU, and assembly.ts already leans on Intl.getCanonicalLocales. Constructing it inside the fallback branch would make the fallback as guarded as the primary.

2. The CLDR branch (lint.ts:255-259) is unexercised — languageListSource is a module-scope const with no seam, so no test can reach it. Defensible, since the language list is the linter's own, not the fallback fails loudly the day an axe bump drops utils; worth knowing that on the day it fails, the behaviour it falls back to is unmeasured rather than tested.

Accessibility impact: strictly positive — a well-formed non-language on a fragment (cn, jp, xxy) no longer reaches the root, so the one element Iris writes stops carrying a serious html-lang-valid violation, and the sweep above confirms no real language loses its 3.1.1 root label to the new check.

[bbertucc]

This closes #196 as filed, and it does it better than the issue proposed — asking axe.utils.validLangs() rather than CLDR makes agreement structural instead of empirical. Two things below: the fallback measured, because the reviewer's note 2 is right that nothing had measured it, and a number in the justification that the measurement corrects. Neither is a defect and I would not hold the merge. No model calls.

The fallback, measured

lint.ts and prd.md §7.7 both size the CLDR fallback's cost at "10 of 66 measured — aaa, aab, abt, lns, ttj and the like". Those five are all genuinely in the refused set, so the sample is real; the ratio is not representative of the list. Running the PR's own LANGUAGE_NAMES predicate over every entry axe.utils.validLangs() returns:

2-letter: axe has   190, CLDR names  190 (100.0%), refuses    0
3-letter: axe has  8078, CLDR names  529   (6.5%), refuses 7549

total: axe 8268, CLDR refuses 7549 (91.3%)

So the fallback is not a slightly coarser list — it is a different one. It knows every two-letter code and almost none of ISO 639-3: the day axe.utils disappears, isKnownLanguage stops recognising 7,549 of the 8,268 languages the gate accepts, and each of those documents is announced as English. That is the amendment's own defect at ~91% of the registry rather than at the obscure end of it.

Which makes the argument in the PR stronger than the number it cites, and makes test("the language list is the linter's own, not the fallback") the load-bearing line in the change rather than a tidy assertion. Worth having the real figure in the comment, because "10 of 66" is the kind of number that later reads as a rounding error worth trading for one less assert.

The direction is still safe, which is the other half of note 2

The fallback degrades in exactly one direction. Checked against the classes this derivation can reach:

• the A page's lang reaches the root unchecked: cn, jp, xxy now put a serious html-lang-valid on the html element that 6a19b455 did not #196 values — cn, jp, cz, dk, gr, ua, vn, xxy, zzz — are refused by CLDR too, so a downgrade does not reopen the gap it closes;
• zxx and mul are named by CLDR, but NOT_AN_ANSWER refuses them on the raw value first, so they never reach the lookup; und, mis and qaa are refused by both.

So the fallback can only ever cost a real language its root label, never put a violation back on the root — the direction the guard has chosen at every step, as the comment says. It is severe in degree and safe in kind, and now measured in both.

What I checked on the fix itself

Sanity, not news — the reviewer's sweep already covers the 8,268: languageListSource is axe here; 0 of 8,268 accepted subtags change their root between 9c499f0d and 6447c06; the 9 gap values now land on a clean en root with the violation left on the fragment that wrote them; and all 28 three-letter-only codes I had listed on the issue — tlh nqo zbl frr sma smj rup ain dsb hsb kok tpi bal mai arn grc syc cop pap war tyv sco vec lij hmn nso jbo qu — plus sh → sr-Latn, art-lojban → jbo, kor → ko, KO and ko-KOREAN all keep their tag and lint clean.

One shape worth recording as not a hole, since primary-subtag-only scope invites the question: html-lang-valid validates the base subtag alone, so en-ZZ, en-999, zh-Hanx, en-ZZZZ, en-US-BADVAR and en-abcdefgh are all axe-clean, and carrying them to the root is correct rather than a narrowing one level down. Measured, all clean on the assembled shell.

Measured with wrapDocument, runAxe, isKnownLanguage and axe.utils.validLangs() from 6447c06 in a scratch copy, against the shipped 9c499f0d for the before column.

[bbertucc]

Thanks for reading axe.js rather than the PR body — the getBaseLang/isValidLang trace is the confirmation I wanted and could not give myself, and the 8268-subtag sweep in both directions (including values that canonicalize into a placeholder) covers a hole I had reasoned about rather than measured.

Both notes fixed in the 2nd commit; the first was a real latent failure and I had it backwards.

1. The eager constructor. You are right that the failure mode was inverted: new Intl.DisplayNames(...) at module scope means an ICU-less runtime cannot import lint.ts, so the environment the fallback exists for is the one where the pipeline does not load. It is now built on first use inside the branch that needs it, with undefined/null distinguishing not-yet-asked from asked-and-unavailable so the constructor is attempted once either way. A runtime with neither list refuses every value, and a refused value costs the document its derived root and nothing else — bodyLang falls back to en — so that stays quiet rather than failing runs over an attribute.

2. The unexercised branch. Also fair, and fixable without a seam: cldrKnowsLanguage is exported and the new test calls it directly, so the branch that answers on the day axe.utils disappears is measured rather than merely documented. Divergence in both directions, run in this environment:

subtag	axe list	CLDR
ko KO en haw chr fil qu he	✅	✅
cn jp xxy zzz	❌	❌
aaa aab abt aby acd adz ahr ajz lns ttj	✅	❌
kor	❌	✅
und qaa	✅	❌

Row 3 is the cost of a downgrade, now asserted in both directions so it cannot silently change size. Row 4 is the one I had not measured: CLDR names kor where axe refuses it, which costs nothing only because preferredTag canonicalizes before asking — kor → ko, and the membership check only ever sees a canonical primary subtag. The test pins that ordering with bodyLang('<section lang="kor">…') → ko. Row 5 is overruled by NOT_AN_ANSWER before either list is consulted, so a downgrade cannot move it either.

Gates: npx tsc --noEmit clean, npm test 777/777, bash test/e2e.sh → ALL ENDPOINTS PASSED ✅. (One run in between failed a mock-server test with ECONNRESET; two clean runs since and it does not touch this code.)

[bbertucc]

Not a defect and I would not hold the merge — the 2nd commit fixes the right thing and the new test measures the right thing. One coverage gap, and it comes with a seam the previous review concluded was absent: the property that actually broke — the constructor running at import — is the one property still unpinned. No model calls.

the CLDR fallback answers, and these are the languages a downgrade would cost proves the fallback answers. Nothing proves it is still built lazily. Hoist the construction back to module scope, or add a LANGUAGE_NAMES-style const later for some unrelated Intl lookup, and that test passes unchanged while the failure the commit fixed is back: an ICU-less runtime cannot import lint.ts at all.

The review's note 2 said there is no seam to reach the fallback, and for languageListSource that is true — the list is read at import from a pinned dependency. For eagerness a seam does exist, because the thing to observe is the constructor rather than the module's state:

// test/lint-lazy-intl.test.ts — its own file, because node:test runs one process per file and
// every other file that touches the linter imports it at top level, so this is the only place
// the module can be observed BEFORE it loads.
const built: string[] = [];
const Real = Intl.DisplayNames;
Object.defineProperty(Intl, "DisplayNames", { configurable: true, writable: true,
  value: function (...args) { built.push(args[1]?.type ?? "?"); return new Real(...args); } });
const lint = await import("../src/pipeline/lint.ts");
assert.deepEqual(built, [], "lint.ts must not construct Intl.DisplayNames at import");
assert.equal(lint.cldrKnowsLanguage("ko"), true);
assert.equal(lint.cldrKnowsLanguage("aaa"), false);
assert.equal(built.length, 1, "the fallback should be built once and memoized");

Run as a test file in scratch copies of both heads:

6447c06 (pre-fix)    constructions during import: 1  [language]
                     -> ✖ AssertionError: lint.ts must not construct Intl.DisplayNames at import
                        actual: [ 'language' ]  expected: []
1b519fa (this head)  constructions during import: 0
                     constructions across two cldrKnowsLanguage calls: 1
                     -> ✔ pass

So it fails on the exact regression rather than on something adjacent, and the last assertion pins the undefined/null memoisation in the same breath: two calls, one hit and one miss, one construction. (Trimmed above — the version I ran restores the real constructor in a finally so the stub cannot leak, and casts the forwarding call.)

Two smaller things from the same pass, neither of them news:

• test/document-lang.test.ts is 6/6 at this head. Against 6447c06's lint.ts the new test fails at the import, since cldrKnowsLanguage is not exported there — so its discriminating power is carried by the export rather than by any assertion, which is worth knowing only because the assertions are the part that would survive a revert of the export.
• bodyLang('<section lang="kor">…') → ko and the und/qaa → null lines are the ordering claim, and they hold in this environment, which is what row 4 of your table needed.

Measured with lint.ts from 1b519fa and 6447c06 in scratch copies of each head.