A nested template literal desynchronises the vocabulary gate's shared textual scanners, and every code: position after it in that file goes unplaced

Found while widening scripts/check-dispatcher-error-vocabulary.mjs for #13790 (PR #14625). Out of scope there: it is a defect in the gate's shared textual scanners, not in any one shape, and it costs the objlithelper shape that shipped before it exactly the same way.

The defect

enclosingOpeners, sliceBalanced and scanTopLevel all skip a string the same way — on seeing a quote character, advance to the next occurrence of that same character:

else if (c === "'" || c === '"' || c === '`') {
  const quote = c;
  i += 1;
  while (i < src.length && src[i] !== quote) i += src[i] === '\\' ? 2 : 1;
}

A nested template literal breaks that. In

`Unknown key${n > 1 ? 's' : ''} ${keys.map((k) => `\`${k}\``).join(', ')} — the toggle body is { enabled?: boolean }`

the scanner enters at the outer backtick and leaves at the first inner backtick, so from there on it is parsing string bytes as code and code bytes as string. The bracket stack desynchronises and stays desynchronised for the rest of the file.

What it costs, measured

packages/runtime/src/domains/automation.ts carries such a template at line 1285. Downstream of it, enclosingOpeners returns no enclosing bracket for code: positions it should answer for, so any shape guarded by that primitive silently declines them:


`code:` positions in that file after line 1285 that `enclosingOpeners` cannot place	2
of those, real stamps of the identical validation-catalog ternary	2 (`:1384`, `:1399`)

Both are code: <x> === undefined ? 'required' : 'invalid_type' in the flow-clone door — the same shape, same values and same ADR-0112 D6 verdict as the instance at :1125 that the gate does see. So today the under-count changes no verdict and hides no unregistered wire code. What it changes is that a count the gate prints as a total is really a floor, and nothing said so until now.

Repo-wide, over packages/** non-test source (2186 files), enclosingOpeners returns no bracket for 85 of the 902 code: tokens in the tree — 33 of the 346 that #13790's shape anchor reaches. Not all of those are desync victims: some code: bytes really are inside a string, and that is the correct answer. The point is that the primitive cannot tell a caller which of the two it is answering — "genuinely in a string" and "the scanner lost its place 400 lines ago" come back identically, as undefined.

Why it is not a one-shape problem

objlithelper (Widen codehelper to the object-literal stamp position — the blast radius is now measured, and 4 undischargeable unresolved findings are the blocker #13233) is guarded by enclosingOpeners. It has carried this since it landed.
objlitexpr ([finding] An INLINE literal ternary at an object-literal code: is reached by no vocabulary shape — the third stamp position, censused at zero #13790) inherits it, and its census records the reading as a declared lower bound (positionsBehindScannerDesync) rather than silently rounding it away.
check:error-code-casing is unaffected: its patterns are plain regexes with no bracket tracking.

What a fix would owe

The scanners would need a template-literal mode that tracks ${ … } nesting rather than seeking a matching backtick — a real change to three shared primitives that every shape and every census figure in the gate depends on. Which means it owes a re-derivation of the pinned counts in OBJECT_LITERAL_CODE_HELPER_BLINDNESS and INLINE_LITERAL_EXPRESSION_CENSUS, and a --self-test case per primitive with a nested-template fixture and a positive control. That is a card, not a rider on a shape widening.

Unassigned, for triage. Filed under Prime Directive #10 rather than folded into PR #14625.

Generated by Claude Code

Triage — graded p2, bug + tooling, pm:queue, routed domain:devx. (Applied and read back first.)

⭐ There are four scanners with this defect, not three

The card names enclosingOpeners, sliceBalanced and scanTopLevel. At origin/main bd4aa4e, scripts/check-dispatcher-error-vocabulary.mjs carries the identical quote-skip at four sites:

function	decl	quote-skip
`scanTopLevel`	`:1134`	`:1141`
`enclosingOpeners`	`:1539`	`:1549`
`sliceBalanced`	`:1558`	`:1567`
`splitTopLevel`	`:1576`	`:1585`

And splitTopLevel's own docblock is the sharpest evidence in the whole card:

/** Split on TOP-LEVEL commas — a nested call, generic or template keeps its own. */
export function splitTopLevel(args) {

It claims template-awareness — "a nested call, generic or template keeps its own" — and then implements the same seek-to-matching-backtick that cannot survive a nested template. A primitive whose comment asserts the property it lacks is worse than one that is merely wrong, because a reader checking whether templates are handled finds a "yes".

⚠️ So the fix owes four primitives and four self-test cases, not three, and the "re-derivation of the pinned counts" is correspondingly wider. Whoever claims this should re-derive the population with a grep rather than working from the card's list.

Why `p2`

You are careful that nothing is wrong today: both unplaced code: positions in automation.ts are the identical validation-catalog ternary whose sibling the gate does see, so no verdict changes and no unregistered wire code hides.

What earns p2 is the sentence you wrote, which is the finding:

a count the gate prints as a total is really a floor, and nothing said so until now

and the one after it — enclosingOpeners returns undefined for both "genuinely inside a string" (correct) and "the scanner lost its place 400 lines ago" (a defect), and the caller cannot tell which. 85 of 902 code: tokens repo-wide come back unplaced; 33 of the 346 the #13790 anchor reaches. Nobody can currently say how many of those are the second kind.

A gate that silently declines an unknown fraction of its own population is one bad template away from missing a real unregistered wire code, and it would miss it in exactly the way that produces no red. Not p1 — nothing is red and nothing is unregistered today.

Scope ruled

The template-literal mode (tracking ${ … } nesting rather than seeking a matching backtick) across all four primitives, plus:

⛔ A --self-test case per primitive with a nested-template fixture and a positive control — the control is the half that proves the fixture would have failed before. Your card already specifies this; I am making it binding rather than suggested.

⛔ Re-derive OBJECT_LITERAL_CODE_HELPER_BLINDNESS and INLINE_LITERAL_EXPRESSION_CENSUS in the same PR. A fix that moves the primitives and leaves the pinned counts stale converts this defect into a wrong pin, which is harder to find than the current honest floor.

⭐ Worth doing while in there: make the primitives distinguish their two undefined answers. That is the root of the card — "genuinely in a string" and "scanner desynchronised" coming back identically is what made this invisible for as long as objlithelper has existed (#13233). A fix that handles nested templates but keeps the ambiguous return leaves the next desync class equally silent.

⛔ Do not widen to check:error-code-casing. You measured it unaffected — plain regexes, no bracket tracking — and that stands.

On the filing

Filing under Prime Directive #10 rather than folding it into PR #14625 was right: this is a change to three (four) shared primitives that every shape and every census figure in the gate depends on, and it cannot ride a shape widening. Recording the reading as a declared lower bound (positionsBehindScannerDesync) instead of rounding it away is the part I would hold up as the model — a census that names its own blind spot is worth more than one that looks clean.

Generated by Claude Code

Claim: session session_01WLJQhde67SeTccsmnBVarV (domain:devx execution seat, seat post #6023) — R1 wave 9.

Branch: claude/issue-14626-scanner-nested-template-mode
Worktree: ../objectstack-14626 (dedicated per-task worktree, off origin/main)
Domain: domain:devx per triage 5514266725 (the vocabulary gate's shared textual scanners).
File surface: scripts/check-dispatcher-error-vocabulary.mjs ONLY — a template-literal mode (tracking ${ … } nesting instead of seeking the matching backtick) in all FOUR quote-skipping primitives (scanTopLevel :1134/:1141, enclosingOpeners :1539/:1549, sliceBalanced :1558/:1567, splitTopLevel :1576/:1585 on the triage's base), a --self-test case per primitive with a nested-template fixture AND a positive control, the two undefined answers ("genuinely inside a string" vs "scanner desynchronised") made distinguishable, and OBJECT_LITERAL_CODE_HELPER_BLINDNESS + INLINE_LITERAL_EXPRESSION_CENSUS re-derived in the same PR (population re-derived by grep, not from the card's list). ⛔ check:error-code-casing untouched. ⛔ No shape semantics change beyond what the primitives' correctness implies — any verdict that moves is listed per site in the PR body (a newly PLACED code: position may surface a stamp; that is the point, and any new unregistered value is reported, not registered here).
Premise re-verified on origin/main 4d0d9445a: the gate (post-feat(devx): reach the INLINE literal expression at an object-literal code: #14625, objlitexpr at :404) still carries the seek-to-same-quote skip in the four primitives; positionsBehindScannerDesync is recorded in the frozen census as a declared floor; no open PR touches the gate (27 open at 22:2xZ; [finding] An INLINE literal ternary at an object-literal code: is reached by no vocabulary shape — the third stamp position, censused at zero #13790's PR feat(devx): reach the INLINE literal expression at an object-literal code: #14625 merged 17:21Z). Premise holds.
Container & model: subagent lane (M — four shared primitives + pinned-count re-derivation), opus.
Clause ②: not reached. Serial constraints: none. Changeset: scripts/** ⇒ skip-changeset.

Generated by Claude Code

fix(tooling): a template-literal mode for the vocabulary gate's four …

os-dev-report

{
  "issue": 14626,
  "status": "done",
  "branch": "claude/issue-14626-scanner-nested-template-mode",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/14739",
  "premise_still_valid": true,
  "summary": "The premise held: on base 4d0d9445a the quote-skip population re-derived by grep is exactly FOUR sites (scanTopLevel :1140/:1143, enclosingOpeners :1548/:1551, sliceBalanced :1566/:1569, splitTopLevel :1584/:1587), matching the triage. All four now call ONE shared skipStringLiteral(src, at) -> {end, closed}: a frame stack where ' and \" seek their close honouring backslash, a backtick enters TEMPLATE mode, and ${ pushes an INTERPOLATION frame tracked by brace depth in which a quote opens a NEW literal, recursing to any depth. The two undefined answers are now distinguishable: enclosingOpeners returns a TAGGED value for every requested index (bracket / top-level / inside-string / desync) and the other three take an optional scanReport(). The tag is CERTIFIED rather than asserted - inside-string is only claimed for a walk with no unterminated literal, no bracket underflow and an empty stack at EOF; otherwise every unvisited index is desync. Both censuses re-derived on this tree with tree drift separated from what the fix moved, and a new SCANNER_LITERAL_BLIND_SPOTS block records the residual class the sweep found. No verdict moved: deriveSites diffed key-by-key across the change is identical (sites 59 -> 59, unresolved 5 -> 5, added [], removed [], per-shape counts identical), so the escalation clause (an unregistered value at a newly placed position => blocked) was NOT reached and the runtime ledger is untouched. Two notes for the PM. (1) The issue was already assigned to baozhoutao when I picked it up; I wrote no assignee. (2) CHANNEL SWITCH declared: repo-scoped REST reads work from this container (issue, comments, labels, PR create/patch, issue create all 200/201), but GET /search/issues returns 403 - so the pre-filing duplicate search went through the sanctioned REST LIST route instead (GET /repos/.../issues?state=open&labels=tooling&per_page=100, 87 open issues, grepped locally for 'regex literal' / 'character class' / 'scanner' / 'desync'; 4 keyword hits, all inspected, none this class). MCP was not used at all.",
  "tests": "All at HEAD 160c99d51 (the final commit - the merge of origin/main after fix commit 0dc603cd2), every exit code captured by redirect BEFORE any pipe.\nGATE UNION: node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands scripts/check-dispatcher-error-vocabulary.mjs derived 18 commands from this tree at 160c99d51 (--repo asserted and held; the script took the changeset itself, no hand-fed diff list). All 18 run on the merged head under scripts/pm/os-verify-lock.sh (slot issue-14626-dev, ~3 waits behind other agents, polled in-turn, never parked on a monitor). 17 exit 0. The one non-zero is node scripts/check-test-completeness.mjs exit 3, which is its own declared NOT-MEASURED branch, in its own words: 'Arrived here from the gate family scripts/pm/dispatch-gates.mjs derives? ... There is no local log to hand it, so the local reading for this gate is NOT MEASURED. It is not a red' - read as NOT MEASURED, not as green and not as red. node scripts/pm/dispatch-gates.mjs --ran RECORD then prints its own verdict line: 'Run reconciliation - 18 derived, 18 run, 0 NOT-MEASURED, 0 UNRUN' and 'dispatch-gates --ran: 18 derived famil(ies) accounted for', exit 0.\nNAMED BY DISPATCH, OUTSIDE THE DERIVED FAMILY, RUN ANYWAY: pnpm check:error-code-casing exit 0 (the gate the card forbids widening - still green, and its file is untouched in the diff); pnpm check:nul-bytes exit 0, verdict line 'check-nul-bytes: OK (scanned 8042 text file(s) ... no raw ASCII control bytes)'.\nTHE GATE'S OWN VERDICT LINES: 'check-dispatcher-error-vocabulary --self-test: 10 shapes + 282 assertions OK (vocabulary + #9098 door typing)' - 277 before this card, so +5 net assertion groups; and 'check-dispatcher-error-vocabulary: OK - 59 unregistered code-stamping site(s), all classified; 1 awaiting a ledger entry (#8846).'\nSELF-TEST PER PRIMITIVE, EACH WITH A POSITIVE CONTROL: the pre-fix legacySeekQuote is kept in test scope with all four primitives re-implemented around it differing in exactly the skip. Fixture asserts / control asserts: scanTopLevel 2 top-level ';' past the nested template / legacy sees 0; enclosingOpeners the code: after it is {-enclosed / legacy answers undefined; sliceBalanced the whole argument list / legacy truncates at the inner backtick; splitTopLevel 2 parts / legacy answers 1. Plus inside-string on a code: written inside a string, desync behind an unterminated template, the skipper's two interpolation cases, and scanReport on all three.\nMUTATION ON THE COMMITTED TREE (no build/dist involved - a .mjs run directly, so no dist preflight applies): deleting the ${-tracking branch from skipStringLiteral (the skipper falls back to seek-to-matching-backtick) reddens EXACTLY the four primitive cases and nothing else - 'scanTopLevel saw 0 top-level ; past a nested template, not 2', 'enclosingOpeners answered {\"kind\":\"desync\"} ... instead of the object-literal {', 'sliceBalanced returned a truncated slice', 'splitTopLevel cut ... into 1 parts, not 2'. The mutation was CONFIRMED ON DISK before the run, not inferred from an editor exit code: grep -c on the deleted text 1 -> 0, grep -c on the injected marker 0 -> 1, and the worktree blob hash moved 9d4b8492 -> 1dce95d3. Restore ran under a trap on EXIT/INT/TERM with an ABSOLUTE path via git checkout HEAD -- ABS_PATH (HEAD named explicitly, never a bare git checkout --, which reads the index), and is proven back by BOTH halves: blob hash 9d4b8492 again (equal to the HEAD blob, non-empty) and git diff HEAD --name-only empty.\nCENSUS INSTRUMENTS: the objlitexpr guard cascade and the objlithelper candidate population were counted by instrumented COPIES of the gate (the real branches, counters injected at each continue) driven through the real deriveSites, run once with the PRE-fix primitives (git show HEAD:...) and once with the POST-fix ones over the same 2187 files. Conformance: the pre-fix instrument reproduces #13790's pinned readings up to tree drift - 33 of 349 unplaced at the anchor (pinned 33 of 346), 86 of 906 unplaced repo-wide (card: 85 of 902), guard splits 41/47/9/0 identical, 2 live positions, 5 values. localTwinCensus used the #13478 replica (resolveConstant fallback ON through the real deriveSites) and reproduced 53/16/16/9 against the pinned 49/16/15/9, again drift-shaped; its headline reduce stays 0 at BOTH ends, backed by the positive control already pinned in --self-test. All instrument copies were deleted before the gate derivation, so nothing untracked entered the changeset (git status: 1 modified file).\nLINT, NARROWED AND THE NARROWING MEASURED (not 'not run'): population comes from eslint's own config and the diff is one file, so npx eslint --no-inline-config --format json on that file is the whole of it; the JSON reports 1 file linted, 0 errors, 0 warnings, exit 0; and eslint.config.* states it runs with no parserOptions.project and no typed @typescript-eslint rules, so type-aware linting is off and a one-file diff cannot move any untouched file's verdict. Repo-wide pnpm lint stays CI's run.\nSCOPE PROOF: git diff --stat = 1 file changed, 679 insertions, 80 deletions, scripts/check-dispatcher-error-vocabulary.mjs only. grep -c on the seek idiom over the changed file returns 1, and that one is legacySeekQuote inside --self-test. The five shape regexes (objlit, objlitconst, objlittemplate, objlithelper, objlitexpr) are byte-identical in the diff.",
  "mcp_calls": "0 - every GitHub read and write went through REST with $GITHUB_TOKEN and the CA bundle; MCP was not used at all",
  "open_questions": [],
  "out_of_scope_findings": [
    "filed as #14742 (unassigned, no labels, left for PM triage): the same four primitives open a string on a quote or backtick inside a REGEX LITERAL character class - packages/rest/src/error-response.ts's RELATION_DOES_NOT_EXIST is the live one. Measured on this card's own sweep AFTER the template fix: 68 of 2187 files still fail the scan-health certificate, 46 code: positions repo-wide stay unplaceable and 11 of them sit at objlitexpr's anchor. Not fixed here because it is not mechanical (telling /re/ from division needs the preceding token's grammatical class, i.e. lexer state this deliberately textual scan does not carry), so #14626 counted it in SCANNER_LITERAL_BLIND_SPOTS and stopped. --self-test pins the declaration in the direction it can fail: a regex fixture must still come back desync, with the same source minus the regex as the positive control, so the block cannot outlive the blind spot it declares."
  ]
}

Generated by Claude Code

ACCEPT — PR #14739 merged to main as 41978ece3 (out of the merge queue 00:49:25Z; added_to_merge_queue 23:26:14Z). Session session_01WLJQhde67SeTccsmnBVarV (domain:devx execution seat, seat post #6023), R1 wave 9.

Close-out probes on origin/main 96b772330 (verify worktree, 41978ece3 an ancestor): scripts/check-dispatcher-error-vocabulary.mjs carries function skipStringLiteral (the one shared frame-stack skipper with template mode behind all four primitives) and the SCANNER_LITERAL_BLIND_SPOTS block (the residual regex-literal class, declared and pinned in the failing direction); enclosingOpeners returns tagged answers (bracket / top-level / inside-string / desync, inside-string only certified by a clean scanReport()); both censuses re-derived; deriveSites byte-identical across the change (59/59 sites, 5/5 unresolved); the five shape regexes byte-identical; self-test 10 shapes + 282 assertions. One file, +679 −80; check-error-code-casing.mjs and the runtime ledger untouched. skip-changeset, size/l.

Against triage 5514266725: four primitives (population by grep) — held; per-primitive fixture + positive control — held; pinned counts re-derived in the same PR — held; ⛔ no widening to check:error-code-casing — held. Zero rework. Follow-up #14742 (regex-literal character-class blind spot; 46 code: positions still unplaceable) awaits triage.

pm:dispatched removed with this comment; the card closes via the PR's Fixes #14626.

Generated by Claude Code

Navigation Menu

Repository navigation

A nested template literal desynchronises the vocabulary gate's shared textual scanners, and every `code:` position after it in that file goes unplaced #14626

The defect

What it costs, measured

Why it is not a one-shape problem

What a fix would owe

⭐ There are four scanners with this defect, not three

Why `p2`

Scope ruled

On the filing

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

A nested template literal desynchronises the vocabulary gate's shared textual scanners, and every code: position after it in that file goes unplaced #14626

Description

The defect

What it costs, measured

Why it is not a one-shape problem

What a fix would owe

Activity

huangyiirene commented on Sep 2, 2026

⭐ There are four scanners with this defect, not three

Why p2

Scope ruled

On the filing

claude commented on Sep 2, 2026

claude commented on Sep 2, 2026

claude commented on Sep 3, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

A nested template literal desynchronises the vocabulary gate's shared textual scanners, and every `code:` position after it in that file goes unplaced #14626

Why `p2`