Skip to content

Fix #587: Strict parsing tests for html structural signatures - #734

Merged
squid-protocol merged 1 commit into
mainfrom
fix/587-html-strict-parsing-take2
Jul 29, 2026
Merged

Fix #587: Strict parsing tests for html structural signatures#734
squid-protocol merged 1 commit into
mainfrom
fix/587-html-strict-parsing-take2

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Summary

Part of #518, closes #587.

Adds strict positive/negative, ambiguity-sweep, and ReDoS regression coverage in
tests/core_engine/test_language_standards_strict.py for all 39 non-None html structural
signatures (54 tests). Found and fixed 8 real bugs, all the same root shape: a \b sitting
adjacent to a non-word character inside a shared alternation group.

Also found and filed a separate, real pipeline bug as #733 before writing any tests: html
is tagged line_exclusive but should be block_exclusive (its own inline comment already says
so), and block_exclusive's own delimiter config is empty/broken -- net effect, <!-- -->
comments are never actually stripped from code_stream for html today. Not fixed here, matching
the epic's established pattern of keeping per-language issues scoped to the rules-dict regexes
and filing pipeline gaps separately (e.g. #691/#694/#697). Tests here are written against html's
real (unstripped) code_stream behavior -- dead_code/macros search for the literal <!--
prefix directly, which is why they still work despite the pipeline bug.

Real bugs found and fixed

  1. branch: *ngIf (Angular) inside \b(...)"[^"]*" -- * is always preceded by whitespace
    in real markup, so the leading \b never fired. Never matched.
  2. comprehensions: *ngFor, identical shape to ci: implement cross-platform matrix testing #1.
  3. safety: pattern="..."/sandbox="..."/rel="noopener..."/integrity="..." all ended
    on " inside a shared \b(...)\b group -- the char after a closing attribute quote is also
    non-word, so the trailing \b never fired. 4 of 7 alternatives were dead.
  4. concurrency: loading="lazy"/fetchpriority="high"/"low", same shape as Bump actions/setup-python from 5 to 6 #3
    (decoding="async" happened to self-heal via a coincidental substring match on the separate
    bare "async" alternative -- fixed for the right reason anyway).
  5. immutability_locks: aria-disabled="true", same shape as Bump actions/setup-python from 5 to 6 #3 (also self-healed via
    "disabled" being a substring of "aria-disabled" -- matched regardless of true/false value).
  6. dependency_injection: <script type="importmap"\b -- trailing \b right after the
    closing ". Never matched at all, unconditionally.
  7. decorators: hidden/inert are true boolean attributes almost always written bare
    (<div hidden>); the pattern required [ \t]*= unconditionally, so the dominant real-world
    bare form never matched. Found while writing this test's own positive case.
  8. reflection_metaprogramming: style="[^"]*;" required a literal trailing semicolon
    before the closing quote -- CSS allows omitting the last declaration's semicolon. Dropped it.

All 8 independently verified: old-pattern-reproduces / new-pattern-fixes for each regression,
including the two "self-heals but for the wrong reason" cases.

Noted but NOT fixed (deliberate scope): every attribute-value pattern assumes double-quoted
values only. HTML permits single quotes too, but double-quotes are the dominant real-world/
tooling-enforced convention, and fixing it would mean touching nearly all 39 patterns rather than
a narrow change -- flagged for awareness, not attempted here.

Verification (all gates)

  • Strict test file: 1502 passed
  • Full default suite: 2446 passed, 1 deselected, 1 xfailed -- no regressions
  • ruff/mypy/dead-key audits clean (3 pre-existing, non-deterministic mypy findings confirmed
    unrelated -- same finding count, different specific files each independent run)
  • golden_crucible via tests/tools/crucible_check.py: real diff found (118 differences,
    confined to exactly 5 real .html files across 3 different corpus repos -- confirmed via
    direct golden_diff.deep_compare(), not the truncated test message). Fixtures regenerated and
    re-verified clean in both venvs.

🤖 Generated with Claude Code

Adds strict positive/negative, ambiguity-sweep, and ReDoS regression
coverage in tests/core_engine/test_language_standards_strict.py for all 39
non-None html structural signatures (54 tests). Found and fixed 8 real bugs
in html's rules dict, all the same root shape: a `\b` sitting adjacent to a
non-word character inside a shared alternation group.

Also found and filed a separate, real pipeline bug as issue #733 before
writing any tests: html is tagged `line_exclusive` but should be
`block_exclusive` (its own inline comment already says so), and
block_exclusive's own delimiter config is empty/broken -- net effect,
<!-- --> comments are never actually stripped from code_stream for html
today. Not fixed here (matches the epic's established pattern of keeping
per-language issues scoped to the rules-dict regexes and filing pipeline
gaps separately, e.g. #691/#694/#697). Tests below are written against
html's real (unstripped) code_stream behavior -- dead_code/macros search
for the literal "<!--" prefix directly, which is why they still work
despite the pipeline bug.

Real bugs found and fixed (8, all the shared-`\b`-next-to-non-word-char trap):

1. `branch`: `*ngIf` (Angular) was inside `\b(...)"[^"]*"` -- `*` is always
   preceded by whitespace in real markup, so the leading `\b` (boundary
   between two non-word chars) never fired. Never matched.
2. `comprehensions`: `*ngFor`, identical shape to #1.
3. `safety`: `pattern="..."`/`sandbox="..."`/`rel="noopener..."`/
   `integrity="..."` all ended on `"` inside a shared `\b(...)\b` group --
   the char after a closing attribute quote is also non-word, so the
   trailing `\b` never fired. 4 of 7 alternatives were dead.
4. `concurrency`: `loading="lazy"`/`fetchpriority="high"/"low"`, same shape
   as #3 (`decoding="async"` happened to self-heal via a coincidental
   substring match on the separate bare "async" alternative -- not a real
   rescue, still fixed for the right reason).
5. `immutability_locks`: `aria-disabled="true"`, same shape as #3 (also
   self-healed via "disabled" being a substring of "aria-disabled" --
   matched regardless of true/false value, which was never the intent).
6. `dependency_injection`: `<script type="importmap"\b` -- trailing `\b`
   right after the closing `"`. Never matched at all, unconditionally.
7. `decorators`: `hidden`/`inert` are true boolean attributes almost always
   written bare (`<div hidden>`); the pattern required `[ \t]*=`
   unconditionally, so the dominant real-world bare form never matched.
   Found while writing this test's own positive case.
8. `reflection_metaprogramming`: `style="[^"]*;"` required a literal
   trailing semicolon before the closing quote -- CSS allows omitting the
   last declaration's semicolon and most real inline styles don't carry
   one. Dropped the requirement.

All 8 independently verified: old-pattern-reproduces / new-pattern-fixes
for each regression, including the two "self-heals but for the wrong
reason" cases (concurrency's decoding=async, immutability_locks'
aria-disabled=true).

Noted but NOT fixed (deliberate scope, not a bug): every attribute-value
pattern in this file assumes double-quoted values only (`"[^"]*"`). HTML
permits single quotes too. Double-quotes are the dominant real-world/
tooling-enforced convention; fixing this would mean doubling nearly every
one of the 39 patterns rather than a narrow, isolated change -- flagged for
awareness, not attempted here.

Verification: strict test file (1502 passed), full default suite (2446
passed, 1 deselected, 1 xfailed, no regressions), ruff/mypy/dead-key audits
clean (3 pre-existing, non-deterministic mypy findings in untouched files
confirmed unrelated), golden_crucible via tests/tools/crucible_check.py:
real diff found (118 differences, confined to exactly 5 real .html files
across 3 different corpus repos), fixtures regenerated and re-verified
clean in both venvs.

Part of #518, closes #587. Pipeline bug filed separately as #733.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol
squid-protocol merged commit 4a23f4b into main Jul 29, 2026
38 of 39 checks passed
@squid-protocol
squid-protocol deleted the fix/587-html-strict-parsing-take2 branch July 29, 2026 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Strict parsing tests: html structural signatures

1 participant