Feature/benchmarks - #47
Draft
multiarc wants to merge 156 commits into
Draft
Conversation
Phase 1: five new workloads + 20 twins, parity contract v2 gates (N3b), golden corpus with export-corpus/verify-corpus verbs, five new benchmark suites. Phases 2-6: Rust (Askama/Tera + Criterion), JVM (JTE/Thymeleaf + JMH), JS (Handlebars/Eta + mitata), Python (Jinja2/Mako + pyperf), Go (text/html/template + templ) harnesses under benchmarks/<eco>/ - each with all 32 controlled+idiomatic cells passing the byte gate/verifier. Phase 8: linux-crosscheck tooling (validate.py, launchers, env capture); full gate sweep green on WSL Ubuntu 24.04 at exact toolchain pins. Measurement/publication runs deferred to a dedicated session; Phase 7 awaits published runs per its go/no-go gate. Errata E1-E3 recorded in docs/spec/records.md. Bug fixes related to type resolution Signed-off-by: Aliaksandr Kukrash <multiarc@gmail.com>
Signed-off-by: Aliaksandr Kukrash <multiarc@gmail.com>
Signed-off-by: Aliaksandr Kukrash <multiarc@gmail.com>
Signed-off-by: Aliaksandr Kukrash <multiarc@gmail.com>
Two strands of work, both driven by gaps the 2026-07-25 run exposed. ## The report (complete) Publish docs/benchmarks/2026-07-25 and withdraw 2026-07-22, whose JS render figures were invalidated by the ConsString rope defect. Present workloads in two tiers rather than protocol order. The boundary is the CLR's 85,000-byte Large Object Heap threshold: .NET strings are UTF-16, so a page crosses it at 42,500 characters, and past it copy throughput collapses 4.5x (measured: 40.3 B/ns at 84,800 B, 8.9 B/ns at 108,680 B) with every render driving a full Gen2 collection. Five workloads sit below it, three above. No other ecosystem has this cliff. Tier order is DERIVED and owned by consolidate.py -- no table in a published directory is ordered by hand. Fix the implied-throughput numerator: it divided by the golden byteLength, which is the NORMALIZED form, understating composed-page by 1.56x and hiding a cell above the plausibility ceiling. consolidate.py now emits summary-tables.md as a second generated artifact, which index.md embeds, so the narrative page transcribes no measured figure by hand. ## The new .NET harness (W1-W7 of 12) benchmarks/dotnet/, structured like the other five ecosystems, replacing src/Heddle.Performance as the cross-stack .NET leg. The old project is still intact and still builds; it is not retired until the remaining work items land. - Corpus relocated to benchmarks/dotnet/GoldenCorpus and all six harnesses plus consolidate.py repointed. Bytes and SHA-256s unchanged; every gate re-verified. - Razor now covers ALL EIGHT workloads, up from one. The "1.37x faster than Razor" claim previously rested on the single least favourable workload, and the gap had to be published as a caveat. - Idiomatic track added for .NET (51 templates, 6 engines), discharging Phase 1 D15 by its own recorded revisit trigger. - All six Heddle render techniques implemented and gated: runtime and precompiled backends across string, TextWriter and UTF-8 IBufferWriter sinks. Only the runtime UTF-8 sink is wired into the cross-stack sweep -- one engine contributes one row -- and it is the fairest row, since the other five ecosystems all emit UTF-8/Latin-1 and never cross the LOH boundary. Three things the harness makes explicit rather than assuming: - Precompiled coverage is DISCOVERED from the manifest. The generator silently leaves uncovered templates un-precompiled and the engine serves them from the dynamic path, so an assuming harness would time the runtime backend under the precompiled name. Real coverage is 4/8 and the gate prints it. - Sinks CHECKSUM rather than count. A counting writer proves a length was computed, not that characters were produced -- the defect class behind both the retired sink suite and the JS rope artifact. Verified by injecting a count-only body and watching the differential fail. - Razor's default HtmlEncoder numeric-escapes non-ASCII, which N5 does not canonicalize; it inflated encoded-loop by 125,000 characters. Fixed by configuring the encoder (contract v2 D2's preferred remedy), not by widening the gate. Gate: 140 passed, 0 failed. Selftest: 138 passed, 0 failed, including N3b byte lengths matching the JS reference exactly on all eight workloads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolves the duplication documented in docs/research/generator-code-sharing/:
every rule the source generator and the runtime engine maintained as two
hand-kept copies moves into one shared artifact, and the fifteen verified live
drifts are fixed.
Phase 0 (gate) — the suite can no longer pass through an unintended
precompiled→dynamic fallback: FallbackGuard sentinel + Strict policy,
RenderViaResolver in registry-only and file-backed modes, registry isolation
with a leakage canary, a gauntlet-crossing corpus sweep, and a seeded-mismatch
meta-suite whose negative control pins the pre-phase-0 world. Known drifts
landed as quarantined red fixtures handed to their owning phases.
Phase 5 — content hash, key↔path derivation, .heddle extension rule,
schema/engine versioning, option names/defaults. Fixes the hash-input mismatch
(BOM'd/UTF-16 templates were permanently StaleContent), the silent key
fallback, the fabricated engine version, and dead item metadata. Claims
HED7018/HED7019/HED7020; replaces the generator's blanket catch with a
per-template error under the Q2.2 fallback-legitimacy ruling.
Phase 2 — one implementation of every byte-affecting shaping machine. Fixes the
WidenToWholeLine clamp drift (the generator was wrong on all three sub-cases);
DocumentShaper.cs 398→118 lines.
Phase 4 — shared Roslyn-free rule tables for numeric kinds, operator legality,
member visibility, hop form, literal formatting and overload rank. Fixes the
ToString("R") round-trip (a build-machine-dependent value difference) and
unguarded binary emission.
Phase 6 — one diagnostic identity across build tier, run tier and editor:
shared catalog, projection, one line-index rule, one alias table. Fixes
forwarded-warning ID/Fix loss and the line-index \r mismatch; LSP gains full
options parity and its default profile aligns Text→Html.
Phase 3 — one rule-core each for extension identity/discovery, function
exports, prop layouts, assignability, member paths and model type names. Fixes
false HED7006, the nested/generic AQN mismatch (discovery *and* formatting) and
the BranchRole mirror. Claims HED7021/HED7023; fixes the runtime's
order-dependent short-name tie in lockstep per Q3.5.
Phase 1 — emitter↔runtime drift. Fixes the needsLocals scan and per-carrier
flags, unknown-@Profile silent acceptance, and DefaultConvertible gaps. Claims
HED7022/HED7024.
Quarantine register emptied: every red fixture earned green by its owning
phase, none deleted or weakened (phase 1's F11 reshaped, documented in place).
Suite 2630 → 4808 passing, 0 failing, 0 skipped on the legs that run.
Known outstanding items are recorded in docs/generator_plan/README.md
(post-implementation review findings + the compile-channel drain gap).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A review subagent reverted a mutation experiment with `git checkout` while the whole program was still uncommitted, resetting PrecompiledGauntlet.cs to its pre-program state. Three pieces were lost; all three are restored here from their specs and red tests: - Phase 3 OQ4 — the prop-layout fingerprint check in the extension-binding arm. The AQN check proves both sides name the same type, not that it still lays its [Prop] slots out the same way; without this a package that re-orders a slot renders into the wrong slots. Vacuous when the fingerprint is absent, per the additive-schema contract. Specified by PropLayoutFingerprintTests. - Phase 5 D2 — both staleness path reconstitutions go back through the shared TemplateKey.ToPath instead of hand-rolled Path.Combine (behaviourally identical; the regression was to the shared-artifact claim, not behaviour). - Phase 6 WI5 — the gauntlet's HED7101 const points at the shared HeddleDiagnosticIds row again, as its sibling in PrecompiledTemplates does. HashFile's decode-then-hash and the AqnFormatter routing had already been restored by the reviewer from its own context; both verified correct here. Suite green: 4808 passed, 0 failed, 0 skipped on the legs that run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
WI1-WI10 verified intact against source and the plan's Implementation record. The hand-restored TemplateKey.ToPath call sites and HashFile body confirmed faithful by their original author. One residue found: 'using System.Globalization;' in PrecompiledGauntlet.cs, whose only consumer was the CultureInfo.InvariantCulture inside the ToHex that WI1 deleted. The revert restored the using without the code. Removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tring
Tranche A + B verified intact. The hand-restored prop-layout fingerprint check
was behaviourally faithful but wrong in its detail string:
- it broke the file's uniform "<Thing> 'name': manifest=X live=Y" shape; and
- it dropped the <none> sentinel. PropLayout.Fingerprint returns null when the
live extension declares no [Prop] at all -- the package-removed-them case,
the most diagnostic one -- and interpolating null rendered 'live=)', which
reads as a formatting bug rather than a finding.
Root cause was in the original work, not the reconstruction: the test asserted
only Contains("prop layout"), so any message shape passed. The detail is now
pinned in full, and a new test covers the null-live-fingerprint path, which had
no coverage before.
Also reworded phase 3's Implementation record, whose '2 skipped' parenthetical
had gone stale once phase 1 un-skipped the last fixture, and added a
restoration note recording what was destroyed and why the drift was invisible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The audit ran under two transient API failures; its in-flight work is complete and verified here. D12.5 was the adversary reviewer's finding: the plan asserted that 'a host that drops a channel turns the suite red', but no test compared the LSP drain against the generator drain -- what shipped was unit tests of the drain rule plus one single-template check. Now closed. DiagnosticCorpusVectors is one shared table asserted by all three hosts (runtime, build tier, editor), so they are compared to each other instead of to three sets of hand-written expectations that can drift apart in exactly the way phase 6 exists to stop. Also: catalog row count corrected 80 -> 82 in the README (the plan record had already been corrected). BOM investigation closed as a non-finding. Two goldens did lose a UTF-8 BOM (Example5_DefinitionWithProps, Example7_FunctionShimCall), but the pre-program baseline was itself inconsistent -- those two carried a BOM and the other six did not. The program normalised all eight to no-BOM, which is homogeneous and matches what Verify emits. Recorded rather than reverted. Suite: 4930 passed, 0 failed, 0 skipped on the legs that run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lse rationale Done inline: the phase-0 subagent died to four consecutive API 529s without starting, and the queue should not stall on a server-side outage. Coverage gates tightened -- both were reported by the verifying reviewer: - The precompiled-corpus gate asserted '>= 25' against an actual 40, so fifteen templates could stop precompiling with nothing going red. Now an exact count, with a comment saying a change to it is a deliberate act. - Five corpus tests carried 'if (dir == null) return;', which turns the whole gate into a silent no-op if the build layout ever changes -- zero signal, reported as a pass. All five now fail loudly with a message saying how to build the corpus. D7's third seed added. The meta-suite had triples for content hash and extension AQN but none for the options fingerprint -- gauntlet step 1, the earliest check, previously unseeded. D7 calls this seed 'wrong fingerprint arity'; a literal arity change would not compile the manifest, so what is seeded is a wrong fingerprint *value*, the observable form of the same condition. Strict throws OptionsMismatch, the sentinel fires, and the unguarded negative control renders identical bytes. D4's rationale corrected. It claimed 'the corpus is the union of the feature templates', and that claim was load-bearing for the whole coverage argument. It is false: feature suites build templates as inline strings, so ~130 feature tests never cross the gauntlet. Criterion 2 is likewise restated as what the suite actually asserts -- exact-count gauntlet coverage over the precompiled corpus, plus byte parity on the model-less subset in both sub-modes -- rather than the corpus-wide byte parity it claimed and never had. The residue is recorded as open, not quietly closed. Suite: 4936 passed, 0 failed, 0 skipped on the legs that run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…delivered No restoration damage: DocumentShaping.cs verified byte-faithful to the pre-program runtime method-by-method, with only the two licensed differences (the safe ApplyRemove/Replace pair, and comments promoted to doc comments). The clamp fix is intact across all three sub-cases, and the observer re-hosting preserves the pre-extraction HED300x ordering exactly. Pin 7 could not have caught the divergence it was written for. Inside the strip machine, BranchKind.Participant and Other are extensionally equal -- both do stripPrev = null -- so no assertion over the working document or the rebased positions can separate them. Deleting the Participant arm reddens nothing. The plan's risk row said as much, but the pin was recorded as though it discharged the risk. The constraint now sits where it actually exists: enum arity pinned so a backslide to four kinds is a red build, plus a recording observer asserting that a [ScopeChannel] non-role chain is reported Participant rather than Other, that no gap crosses it, and that events arrive classified -> gap -> completed (the ordering every re-hosted diagnostic depends on, previously unpinned at machine granularity). The residual unpinnable case is stated in the test's own doc comment. Nine surviving mutants closed. The landed pins covered the three classification *classes* but never an *equality*: six boundary comparisons in ShiftBySkippedTokens, two in ShiftListsAfter's chain-enclosing arm, and the TrimHiddenRemnantLines already-removed-span guard. Pin 2's 'two comments on one line' row did not constrain the guard it named. Two other survivors are provably extensionally equal and are recorded as such rather than 'fixed'. Q2.1's pin was half-missing: WI8's criterion demands red if *either* side reintroduces the empty-default-chain skip, but the only test drove the generator half. The runtime half is now pinned in the pass-order lockstep suite. WI5's benchmark recorded as undelivered rather than argued. Its done-when demanded allocation-neutrality 'proven, not argued'; it was never measured and the omission was disclosed nowhere. A credible paired run needs a mutation held across a multi-minute BDN run -- the posture that caused the contamination these audits exist to clean up. The standing argument is now labelled as an argument, and it corrects WI9's claim: render allocation is unchanged by construction, but the compile path did gain three allocations per body (one display class, two capturing delegates) that WI9 glossed as zero. Suite: 4986 passed, 0 failed, 0 skipped on the legs that run (was 4936). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…le floor Docs-only for the plan: phase 7 is authored as proposed, not started, because its territory (src/Heddle.Tests/TestTemplate/) was under concurrent audit. The principle lands as a standing rule, not just a plan's motivation: testing-standards.md gains a Test-input single-sourcing section headed "a duplicate test input is a duplicate rule one level up", with ledger entry E9 recording it as additive and as the mechanism E8's corpus-contribution bullet had asked for without specifying. Load-bearing design decisions: a shared MSBuild props file rather than a new csproj or embedded resources (the contract under test is file-based -- RootPath, the .heddle rule, HashFile's FileStream+BOM decode); a compile-checked intent table on two orthogonal axes (Tier x Render) with mandatory justification and bidirectional completeness gates; and set-equality gating rather than a count, because a count is still rubber-stampable by editing one digit whereas set equality can only be made green by naming the file and writing why. Also tightened here, found by the phase-7 survey and missed by the phase-0 audit: CorpusDifferentialTests asserted a ">= 40" floor against an actual 62, so a third of the corpus could disappear silently. Same anti-pattern as the ">= 25"-against-40 gate fixed in 45154fd, which is itself the argument for phase 7's D5 replacing both with set equality. Suite: 4986 passed, 0 failed, 0 skipped on the legs that run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ants Note on provenance: part of this audit's work (the TemplateEmitter fold, GeneratorNumericTableAdoptionTests, and an intermediate LiteralRoundTripTests) was swept into commit 8f9c1fe by an orchestrator "git add -A" while the audit was still running. No mutation residue reached any commit -- verified across all eight. This commit carries the remainder. The duplicate numeric table is folded. TemplateEmitter.IsImplicitNumericWidening was a live second copy of the C# 10.2.3 widening table with five call sites and ZERO test references -- the record's claim that "the lockstep test covers both existing copies" was false, because that test transcribes the runtime's body only and phase 1 never adopted the shared table. It now delegates to NumericTable.IsImplicit via SymbolFacts.ToNumericKind, with the deleted body kept verbatim as a characterization pin swept exhaustively over SpecialType squared in three directions. Two record claims corrected in place. Drift #9's guard strengthened, and its limit stated. Reverting G17/G9 to "R" reddened exactly one test -- a literal string assertion -- while all four round-trip legs (60,000 values plus corners) passed under the bug, because "R" IS shortest-round-trippable on CoreCLR. The guard is now a format-identity assertion, and the same mutation reddens 23 cases across 5 methods. What stays unverifiable here is the defect itself: "R" only mis-round-trips under a .NET Framework host, so G17's sufficiency can only be observed on net48. A revert is caught everywhere; the Windows check remains the only thing that closes #9. Four surviving mutants, each a real hole, each now killed and re-mutated dead: - Cast-pinning deleted entirely reddened nothing. D10's whole mechanism was unobserved -- only the ranker's refusals were covered. Harmless for built-ins today, but phase 3 routed arbitrary host export signatures through the same path, where WI10's 0-of-480 measurement explicitly does not carry. - Unary and ternary guards deleted together reddened nothing, despite the success criteria naming all three arities. Real divergences behind it, including a ternary that is CS0173 in the consumer's build. - The shift row's degrade arm deleted reddened nothing. C# has no <<(int, long), so "Count << Big" was CS0019 in the consumer's build precompiled and rendered at run time -- precisely the class this phase exists to close, on a row no fixture reached. - The string-+ row promoted to Supported reddened nothing, because every fixture partner's ToString happened to match C#'s overload choice. Also: the OQ3 internal-property criterion rested on a fixture no test used (Order.Secret was added for it and never referenced), so the criterion was carried by a unit pin over a Heddle.Tests-internal type -- a different accessibility situation from generated code in a third assembly. Now covered on both tiers. And bool & bool? is confirmed contained, unchanged, and pinned for the first time, with the contrast that bool == bool? does get positioned HED1008. Suite: 5108 passed, 0 failed, 0 skipped on the legs that run (was 4986). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Q7.* questions were written into the phase-7 plan and never reached docs/generator_plan/open-questions.md, which is the authoritative register. That bookkeeping failure is the reason this section exists. Sweeps every unresolved question the two post-implementation reviews and the six phase audits raised, not just phase 7's: Q7.1-Q7.4 (corpus scope, benchmark convergence, written artifacts, stage-5 boundary) and Q8.1-Q8.8 (the overload-tie silent degrade against the match principle, the P1 gate mechanism, the two generator-only shared cores, [ExportExtensions], StripGlobal's hard-coded assembly name, the compile-channel drain, native-expressions deviation 1, and net48/net6.0 verification). Each carries a stated default so work can proceed under a named assumption if a ruling does not arrive -- the convention the resolved sections already use. The file header no longer claims everything is resolved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Q7.4: all migration stages including 5 land inside phase 7 -- the D4 coverage residue is closed completely rather than left as a tail; each stage keeps its own byte-neutral gate so the open-ended scope is bounded by per-stage acceptance. Q8.1: the overload-tie silent degrade becomes a build error, HED7025. The generator must not stay silent about an illegality it has already proved. Accepted consequence: a project with an ambiguous overload call that builds green today starts failing -- the same posture as phase 5's emitter fault and phase 3's HED7021. The side condition is mandatory (report only when no argument estimate is Unknown), and phase 4's fixture must stop pinning the silence. Q8.2: raise MinSupportedSchemaVersion to 4. The break ships as declared at version 2.1 with no compatibility shim, but the gate must reject cleanly rather than advertise a support window the metadata cannot honour -- and the rejection must be demonstrated by a manifest fixture built at the old schema, not asserted. Q8.3: fold the runtime onto ExtensionRegistrationRules and TypeSpelling. Acceptance is that mutating each shared rule reddens at least one RUNTIME test -- the property whose absence made them transcriptions rather than sources of truth. Q8.4: model [ExportExtensions] in generator discovery. Acceptance requires a fixture using an extension in an assembly without the attribute, since no test uses such an assembly today. Q8.5: fix StripGlobal's hard-coded assembly name, TDD -- the reproducing fixture lands red first, because the defect's whole character is that no fixture reaches it. Unruled and standing at their defaults: Q7.1 (editor templates stay separate), Q7.2 (props serves Heddle.Performance; template sets do not converge), Q7.3 (delete the written artifacts), Q8.6 (compile-channel drain stays unscheduled), Q8.7 (native-expressions deviation 1 fixed in a docs pass), Q8.8 (drift #9 stays unclosed until the Windows net48 run). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Q7.1: keep the editor-tier templates separate. No EditorOnly tier; the intent table's three-value Tier axis stands. Q7.2: leave Heddle.Performance alone entirely -- a benchmark effort is mid-flight there. This overrides the plan's default, which had the shared props file serving it so the path-traversal helper would die everywhere. It now serves the four test projects only, and the benchmark project keeps its traversal helper. Recorded as ACCEPTED RESIDUE in both the register and the plan's non-goals, explicitly so a later reader does not mistake it for an oversight and 'helpfully' fix it: the failure class D2 eliminates is gone from the test suites and survives in the benchmark project until the benchmark work settles. Q7.3: relocate the six written artifacts rather than delete them. Also overrides the recorded default. WI4's done-when now requires the artifacts still to exist at their new location, not merely to be absent from the glob. With Q7.4 already ruled, phase 7's open-questions section is fully resolved and now points at the register as authoritative. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Q8.7's ruling, widened from one wrong sentence to a planned sweep. Docs-only; proposed, not started. The survey found the anchor defect was not alone. native-expressions.md carries FOUR false normative claims, not one -- and because the authority convention points at that document first, each is a latent bug rather than a typo: - user-defined operators are documented as honored; they are honored for arithmetic/relational/equality/coalesce and NOT AT ALL for bitwise, shifts or any unary, where FailBinary/FailUnary fire before the factory; - the shift row documents an int right operand, but any integral is accepted and converted, so a long-shift expression compiles here and is CS0019 in C# -- the doc UNDERSTATES, so matching code to it would break working templates; - lifted operands are documented as covering relational AND equality exactly as C#, but lifting is numeric-path only. The sharpest inversion: the tree's most accurate description of native-expression semantics is NativeOperatorRules.cs, whose comments document every divergence correctly beside the verdicts that encode them -- while the document the authority convention designates is the least accurate. Hence D3 narrows the convention: a normative doc outranks code only for claims that are gated or dated-verified; an ungated claim is evidence of intent, not authority. Diagnostics are the clearest split. All 24 HED70xx are triple-gated and drift-free across three phases, while 20 of 82 shipped IDs appear in no published document and HED1xxx -- whose registry-designated owner IS native-expressions.md -- is 2 of 17. HED1008, the error the anchor defect is about, is documented nowhere. Stale citations are not where anyone expected: all 830 file-and-line references sit in generator_plan and research (86 past EOF), and the published docs and specs carry zero. So class S is scoped as a gate only, never a hand-fix. Three code defects escalated rather than fixed, per D2 (the sweep corrects documents, never code): the id-less nullable-bool bitwise error, floor of an int reaching HED1013, and a shipped sample still passing the removed Name item metadata. Version sweep assigned to Q8.2, not here: 2.1 IS the declaration of the binary break and must ship atomically with MinSupportedSchemaVersion = 4 (verified still 1) and the old-schema fixture. The surface is 13 must-change files plus 5 coupled, not nine, and the riskiest is the VS Code extension's PINNED_VERSION, outside any check today. Four new open questions recorded with defaults: Q8.9 (convention placement and retroactivity), Q8.10 (phase 7 slip), Q8.11 (centralise the version elements), Q8.12 (the sample's stale metadata). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…sions] Both ruled by the user after the post-implementation reviews. Phase 1's audit is editing this tree concurrently; its files are deliberately excluded here and land separately. Q8.3 -- the two "shared" cores had no runtime caller, so they were transcriptions rather than sources of truth. Baseline reproduced before fixing: mutating ExtensionRegistrationRules.Resolve's Conflict verdict killed only the direct table assertion and no behavioural runtime test, exactly as the reviewer reported. Now TemplateFactory.AddExtensions resolves every name collision through Resolve and switches on the verdict, LoadExtensions sorts by OrderingKey, and ReflectionHelper.ResolveType drives TypeSpelling.TryResolve through a new reflection ITypeLookup adapter -- deleting its duplicate ExtractGenericArguments, TryFindMatchingAngleBracket, SplitTopLevelArguments, ResolveGenericType, ResolveArrayType and the tuple regex (-185 lines). ResolveSimpleType stays deliberately: the assembly-scan index, the dot-to-plus retry ladder and the ambiguity rule are the reflection tier's own universe, not grammar. Acceptance was that mutating each shared rule must redden a RUNTIME test. Nine mutations, nine runtime deaths, all reverted -- table in the plan record. The new ExtensionRegistrationPrecedenceTests assert the three verdicts through the public AddExtensions seam; their absence is why the transcription was invisible. ExportBookkeeping had no test at all and now has nine, placed in Heddle.Tests on purpose so mutating it reddens a runtime leg. The fold surfaced two real divergences, both fixed in the shared file so the tiers move together: a one-element tuple is legal and reflection has always resolved it, so the parser's two-element floor was a build-tier-only refusal the extraction itself introduced; and a whitespace-padded top-level spelling now resolves on the run tier where its own dispatch threw. Q8.4 -- the generator scanned all referenced assemblies while the runtime only scans those carrying [ExportExtensions], so it precompiled extensions the live registry can never resolve. ExtensionBinder.CollectExported now reproduces TemplateFactory.ObtainExtensions: the engine assembly whole and unconditional, any other assembly only what the attribute names or the whole assembly for the parameterless All form (short-circuiting the rest exactly as the runtime's break does), and nothing without the attribute. Nested-container discovery survives on both paths -- asserted, because Q8.4 must not undo F1. TDD: the fixture went red first and for the right reason -- exactly the three narrowing assertions failed while the eight must-still-work assertions already passed. No rendered byte changes and nothing that renders today stops rendering: a bound-but-unexported extension produced a manifest row the gauntlet already rejected on every request, so the dynamic tier was already serving it. The change removes dead precompiled output and a permanent per-request fallback. Two verdict changes: a bodied call to an unexported name is now HED7006 at Error, converting a first-render failure into a build failure; a bodiless call degrades quietly with identical bytes. Two fixture debts fell out and were paid, both making an implicit declaration explicit: probe compilations declare their exports through a single-sourced harness helper rather than copying the attribute into ten probe strings (testing standards, test-input single-sourcing), and the integration suite declared nine extension types its own tests exercise and never exported. Neither fix is window-gated; both judgements are recorded in breaking-windows.md. Q8.4 is defect repair under the match principle -- no correct build regresses. Q8.3's two items are widenings: one repairs build-tier drift, the other widens accepted input with no resolved type changing, and depending on ResolveType throwing for padded input is not a dependency the contract offers. The phase-3 record's B4 row is corrected: TypeSpelling was a re-implementation at landing, not the split it claimed. Suite: 5206 passed, 0 failed, 0 skipped on the legs that run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…closed 24 mutants, 24 killed, 0 survivors. The record's factual claims all hold; what did not hold was the coverage two of them implied. BodyModelRules had no Release enforcement -- the emitter's only link to it was a Debug.Assert, and the table's Chained column had NO consumer anywhere. The emitter now DERIVES each nested body's build context from the table row, so it is load-bearing in Release, and the tautological adoption theory (which asserted the table against its own InlineData and never ran the emitter) is replaced by tests that run the generator and read the consequence off emitted source. Byte-neutral. Two tests pinned method NAMES, so a re-implementation under another name passed. Replaced with pins on the INPUTS a re-implementation must read -- proven by adding a renamed leftmost-only probe and watching it go red. The participant-scan lockstep was 6 hand-picked rows where the matrix promised a whole-corpus sweep; it now sweeps all 62 corpus templates, pins by name the six that provision a frame as an anti-vacuity check, and carries an empty named over-provision allow-list. Exact agreement on all 62 today. The test matrix's six "corpus guardrail entries" had zero fixture files, and the claim was wrong twice over: CorpusDifferentialTests has no fixture set of its own, and none of the six shapes was ever added to the corpus it sweeps. Corrected rather than backfilled -- each guardrail now names the differential or lockstep asset that actually pins it (rendered bytes on both tiers, a stronger gate than the corpus's classification pin), and three of the five are stated as unable to be corpus files at all. Eleven further rows naming never-created files now name what exists. Root cause was a naming class of error, now stated at the top of the matrix: the integration suite holds no .heddle files, so a "fixture" there is an inline string keyed by a views/<stem>.heddle path. Q8.5 is closed as phase 1's own D14 artifact: StripGlobal is deleted and the branch arm's manifest row takes Info.BareTypeName/Info.AssemblyName. Byte-neutral, and honestly labelled -- the defect is unreachable by test because the arm gates on IsEngineAssembly, so the guard is a source-shape pin rather than behavioural coverage. WI7's actual fix had no test at all; the new CallTargetAdoptionTests exports a function named raw from a synthetic compilation and requires the emitter to bind the extension and emit nothing into the container. Swapping the classifier's arms reddens it. The F11 fixture is narrowed honestly: clause 3 was a verbatim duplicate of a lockstep test and is dropped; clause 2 was decorative (leftmost participants the old buggy probe already handled) and is replaced by the observable half of the same drift, the per-carrier flag asymmetry. Reverting the flag OR now reddens this fixture, not only its sibling suite. Five mutants were killed only by rule-level tests with every differential and golden suite green. Recorded in the plan because it locates blind spots rather than closing them: no fixture inherits out:: from a base layer, no fixture has a foreign-origin region-fill candidate, and NO extension anywhere carries [EncodeOutput] and [NotEncode] together. Q1.2 confirmed: no present coercion-rail mismatch. Both additive APIs are in the public-API golden and now genuinely exercised -- BindDefinitionOverloadTests is the first test of the "existing overloads unchanged, both flags equal implies identical binding" claim, which the golden only covered at signature level. Suite: 5208 passed, 0 failed, 0 skipped on the legs that run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Q8.13 the value-path coercion rail has no byte-level fixture (shape plus render-path only); Q8.14 RenderTypeRules' fourth truth-table row is unreachable from any real extension, so [EncodeOutput]+[NotEncode] is either meaningful and needs a fixture or incoherent and should be an error on both tiers; Q8.15 two generator tests fail intermittently under concurrent multi-TFM runs, which matters because every mutation result in these audits rested on a test going red for the right reason; Q8.16 RegionTests.LocationOffsetOf returns a hard-coded 0, so it reads as a position assertion while asserting nothing; Q8.17 SymbolTypeIndex.Cache pins every Compilation for the process lifetime. Each carries a stated default. Q8.15's default is to stabilise before further mutation work, on the grounds that a flaky gate is a broken gate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twenty of twenty-one post-implementation questions are now ruled. Only Q8.6 (the compile-channel drain) stands at its default, deliberately. The substantive correction is Q8.12. The register recorded Q5.1 as "Name removed per the recommendation" and phase 5 implemented that removal. The user's instruction is that removal was never the ask -- only wiring Precompile was. So Name is being restored as real optional custom key mapping. Name was dead code (CompilerVisibleItemMetadata, never read), so removing it changed no behaviour and the sample's Name="BuildReport" was always ignored; the defect was that the feature was never wired, not that the metadata existed. Restoring it is the smaller fix and closes Q8.12 as a side effect. Q8.10 rejects phase 8's D9 as designed: documentation has a different job from a test fixture, and byte-identity with a corpus entry is not a property worth buying. No @include: from corpus templates, no corpus intent rows for doc examples, and phase 8 no longer blocks on phase 7. The false doc/test coupling that already exists gets deleted rather than formalised. Q8.14 opens a genuinely new surface: a declaration-side analyzer, so an extension author sees an incoherent [EncodeOutput]+[NotEncode] pairing in their OWN build (warning, HED7027) rather than only their consumers seeing it in theirs (error, HED7026). That is Q6.1's early-surfacing principle applied to authorship. Q8.15 forbids the easy answer explicitly: a retry attribute is not an acceptable resolution for the two flaky generator tests, because it preserves exactly the property that makes a surviving mutant invisible -- and every mutation result in these audits rested on a test going red for the right reason. Q8.17 asks for an operational contract rather than a size cap: a real operation API, observable capacity, and eviction in which AGE participates, motivated by the edit-and-edit-back case where a cached entry is identical yet old. Q8.11 adds strong-naming for our own unsigned assemblies alongside centralised versioning, so the CS8002 warnings stop; third-party unsigned references are handled explicitly rather than blanket-suppressed. Q8.9 puts the documentation mapping into cross-cutting-decisions. Q8.8 and Q8.13/Q8.16 take their defaults or straightforward yes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four ruled items plus one build-order fix, landed by three concurrent agents on
disjoint file sets. Committed together because they share the diagnostics
registry and docs surfaces, and splitting those by hunk would risk a broken
intermediate commit.
THE MOST IMPORTANT FINDING IS NOT ANY OF THE RULED ITEMS.
Wiring Name revealed that NONE of the three HeddleTemplate item metadata worked
from a real project. Heddle.Generator.targets restated each one as
<Key>%(HeddleTemplate.Key)</Key> inside an Include="@(HeddleTemplate)" transform.
The transform already copies every metadatum, and outside a target a cross-item
%() reference evaluates to empty -- so each element OVERWROTE the copied value
with "". Key was inert. Precompile="false" was inert. Nothing noticed because
every test suite injects build_metadata.* directly and never crosses that file.
That means phase 5's Q5.1 deliverable -- "wire Precompile properly" -- did not
work end to end, and its acceptance tests could not have shown it. The user's
instinct that Name should never have been removed was right for a deeper reason
than the one recorded: the feature was not broken because the metadata existed,
it was broken because the targets file nulled it. Fixed by deleting the
restatements, gated structurally by a set-equality test that also refuses any
future restatement, and behaviourally by the sample.
Q8.12 -- Name restored as optional custom key mapping, implemented as a second
spelling of Key so it shares every downstream rule rather than acquiring parallel
ones. Full participation in HED7002 duplicate and HED7003 case-only-twin checks;
HED7018 out-of-root deliberately SUPPRESSED by an explicit Key or Name, because
that warning's premise is that the flattened key was not asked for. HED7028 was
NOT claimed: both new faults are instances of HED7004's existing fault class at
the same severity, position and remediation, so its message was generalised
instead. A second fix was needed to keep the sample's golden to one line: emitted
#line directives named the registration key, so an explicit Name pointed every
mapped span at a path that exists nowhere. #line now names the root-relative
path -- byte-identical where no explicit key is set, which is why no Verify
snapshot moved.
Q8.2 -- MinSupportedSchemaVersion = 4, and the rejection is DEMONSTRATED. The
fixture compiles a manifest against a reference facade declaring the
pre-schema-4 surface under the real assembly's identity, public-signed, with the
real Heddle excluded from both compilations' reference sets -- so the IL genuinely
carries a MemberRef to .ctor(string, string) that cannot bind to the current
3-arg form. That is exactly what the old test could not be: writing
new PrecompiledExtensionBinding("a","b") binds to the 3-arg ctor and passes null,
producing a new-schema call wearing an old-schema shape. Three arms, including a
control that admits the SAME BYTES at schema Min and observes the
MissingMethodException -- so "the gate prevents a startup crash" is evidence
rather than narration, and 4 is shown to be the exact boundary.
Q8.11 -- one VersionPrefix (2.1.0) replacing nine per-project elements, plus
fourteen kinds of version statement now gated by 17 cases. Two things the survey
had missed: LspServer.InformationalVersion was a live drift ("1.0.0" reported by
heddle-lsp --version for the whole 2.0 line, guarded only by a test comparing it
against itself), and the beta job's --version-suffix had to lose its leading dash
or compose 2.1.0--beta.N. All CS8002 warnings gone: Demo.Models, Demo.Wasm and --
found by the new gate rather than the warning -- LanguageServices.Tests.Corpus,
whose csproj already carried a comment claiming it was signed. Scriban is handled
by a declared accepted-unsigned list, with the honest bound recorded: Roslyn has
no per-reference CS8002 suppression, measured, so the mechanism is keyed on the
reference rather than scoped to it.
Q8.1 -- HED7025, the overload-ambiguity build error. BindOutcome is propagated
out of both binders instead of collapsing to null, and the side condition is
implemented as an early return placed BEFORE OverloadRank.Bind runs, so no
verdict is ever computed for untypeable arguments and then suppressed. Three
tests pin the silent-degrade half. One residue found and pinned: the name-keyed
model answers IsReferenceAssignable false by construction, which was harmless
while it could only degrade and is sound now only because no shipped built-in
parameter type is a reference type other than String/Object -- so a test pins
that parameter-type set. The quarantine fixture is reshaped a second time, and
this reshape REVERSES an assertion the first added: phase 4 pinned the build
tier's silence as though silence were the fix.
Q8.16 -- RegionTests.LocationOffsetOf no longer returns a hard-coded 0. It
recomputes the anchor from what the diagnostic reports, asserting the right file
and that line/column agrees with the raw span. Mutation-proven: span shift, line
drift and wrong-file all redden it. A second decorative assertion in the same
fixture (Assert.NotNull after an Assert.Single that already guaranteed it) was
replaced with a real both-tier offset twin.
Q8.20 -- the missing build-order edge. CorpusResolverSweepTests reads
src/Heddle.Tests/bin/** with no ProjectReference ordering it, which was
reproduced as a real failure. Added with ReferenceOutputAssembly="false" --
build order only, deliberately not a compile-time reference, because that suite
hands hand-filtered reference sets to compilations it creates and loads the DLL
by path at run time.
Mutation testing across all three items: 27 mutants, 26 killed, 1 survivor
classified as provably extensionally equal (a null-coalesce whose two orderings
are reachable only when the operands are Ordinal-equal). The load-bearing mutant
is the one that turns the old-schema fixture INTO the optional-parameter
substitution -- it reddens the suite, which is what proves the fixture is not
that substitution.
Breaking-window dispositions recorded for Q8.2, Q8.11 and Q8.12, plus a 2.1
as-shipped record. Q8.2's is the interesting one: the break ALREADY SHIPPED in
2.0.0, because schema 4's optional parameter removed the 2-arg ctor then. What
2.1 changes is only whether the engine says so or crashes.
Build: 0 errors, 0 CS8002 (was 8). Suite: 5312 passed, 0 failed, 0 skipped on
the legs that run. All 10 samples run and compare-golden clean, with only
codegen-t4-successor's golden changed and reviewed line by line. The eight Verify
snapshots differ by exactly the engineVersion line. net6.0 and net48 remain
unrun on this box.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Promoted from the phase-5 plan's implementation record to the README findings, because it invalidates a DELIVERED acceptance claim rather than an unverified one. Heddle.Generator.targets restated each HeddleTemplate metadatum inside an Include="@(HeddleTemplate)" transform, and outside a target a cross-item %() reference evaluates to empty -- so each element overwrote the copied value with "". Key, Precompile and Name were all inert. Phase 5's Q5.1 deliverable did not work end to end, and its acceptance tests could not have shown it: every suite injects build_metadata.* directly and no test crosses the targets file. Also reframes Q8.12 -- the record said Name was dead code and removal was harmless, which was true as far as it went, but all three were dead because of the targets file, not because the metadata existed. Standing lesson recorded: a build-surface contract verified only through injected analyzer-config values is unverified. Where a claim depends on MSBuild evaluation, something must actually evaluate MSBuild -- which is why the fix is gated both structurally and through the sample gallery. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Q8.18 dissolves: verified in source that precompiled function calls are statically bound (NativeExpressionWriter emits a direct shim call from a build-time table; PrecompiledRuntime never consults options.Functions at render). So the build-time inventory is the only scope that CAN be correct for what precompiles, and a host adding overloads is handled by the gauntlet degrading to the dynamic tier. The escape hatch was verified real: Precompile="false" continues before key derivation and emit. Q8.24 closed as invalid -- precompilation shipped in 2.0.0, so no 1.x manifest has ever existed and the reason-code change has no population to affect. Also recorded plainly: no option was added for manifest rejection. PrecompiledMismatchPolicy is pre-existing 2.0 API and phase 0 depends on Strict; the entry read as though proposing a switch, which it was not. Q8.25 is a CORRECTION to work landed in daf7f91. Name was implemented as a second spelling of Key -- an override -- so a named template stopped resolving by its path. It must be ADDITIVE: the path-derived key is kept and the registered name is added, so imports by either spelling resolve, with a warning (HED7028) where a named template is imported by path. This also invalidates the HED7018 suppression recorded under Q8.12, which must be re-derived rather than assumed to carry over. Q8.19 collect all errors if contained; stop and report the cost if it needs the body walk restructured. Q8.26 leave as is, no TreatWarningsAsErrors. Q8.27 prefer absolute paths and mark relativity where relative is genuinely right, rather than converting everything. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Corrects the Name implementation landed in daf7f91, which made Name "a second spelling of Key" -- an override, so a named template stopped resolving by its path and drew HED7011. The ruling is that Name is an additional name register for imports, not an override: nothing that resolved before may stop resolving. Additivity is now structural rather than conditional. The import map is built in two passes and the ORDER is the mechanism: pass 1 registers every template's key, pass 2 adds each Name alias on top, skipping any spelling already present. A registered name therefore cannot displace a key. Name was removed from DeriveKey entirely and a new DeriveName owns it. TDD: ANamedTemplateIsStillImportableByItsPath was written first and reddened with exactly the HED7011 the defect produces. BothSpellingsResolveInOneCompilation then pins that both resolve in one run, so an implementation that merely swapped which spelling works cannot pass. Resolution is asserted by the imported definition body turning up inlined in the importer's pieces -- @<< pulls in definitions, not text, so "no HED7011" alone would also be satisfied by silently dropping the import. HED7028 (Warning) advises where an import resolves through the key of a template that has a registered name. A new id was right here where landing 1 was right to decline one: every other HED70xx key diagnostic reports something UNUSABLE, and this reports something that works, at a different severity and position. Four of six Q8.12 decisions changed once re-derived off the additive premise rather than adjusted: - HED7018 out-of-root: Key only. An additive Name leaves the path-derived key in place and still out of root, so the warning is still about something real. - HED7002 duplicate and HED7003 case-only twin: keys only. A name registers no manifest row and is never a runtime registry lookup. - Key+Name is no longer a conflict at all -- two names for one template. That HED7004 arm is deleted; the name arms that remain cover the two ways a name is genuinely unusable. The #line key/file separation stands, and the sample changed (below). The breaking-window disposition was corrected IN PLACE rather than appended to, so the normative document does not carry two dispositions: the rename clause now applies to Key alone. Name moves nothing -- not the key, the manifest row, the class name or #line -- so it cannot break a call site or stop an import resolving. Q8.27: absolute where it costs nothing, relative-and-labelled where it does. Outside HeddleTemplateRoot there is no anchor and the old fallback was the bare filename -- unopenable and colliding across directories -- so that now emits the template's own absolute path, at zero snapshot cost (the affected snapshots use synthetic relative paths and gained the views/ prefix they should always have had). Under the root the form stays root-relative, because absolute would bake this machine's layout into the sample golden and any rooted snapshot. Every generated file now states which form its #line directives use, directly under the auto-generated marker. The sample now demonstrates Name the way it actually works. Name="BuildReport" on report.heddle became decorative once the key survived, so the entry class reverts to Templates_Report and a new import-only partial carries Precompile="false" Name="Banner", imported as @<<{{Banner}}. Rendered output is byte-identical. This also repairs a gate that had silently died: finding 10's behavioural half was "Name renames the class Program.cs calls", which stopped gating anything the moment Name went additive. Proven by deleting Name="Banner" and watching the real MSBuild build fail with HED7011. Mutation testing: 16 mutants, 15 killed, 1 survivor classified extensionally equal with proof (an early-out whose absence routes through the collision guard to the same observable state). Restoring the override kills 12 tests. One real gap was found by mutation and closed: dropping the HED7028 per-import dedup survived, because no test imported one spelling twice. Harness note recorded for whoever mutation-tests next: rsync -a preserves mtimes, so a reverted source can look older than the mutant's build output and MSBuild skips the rebuild. That produced one false failure, traced and fixed by touching after revert. Suite: 5348 passed, 0 failed, 0 skipped on the legs that run. All 10 samples run and compare-golden clean. README intro refreshed to the real count and to the HED7018-HED7028 range. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Q8.28/Q8.29 are one defect from either end: an opted-out template is still a participant in the import graph, so its key and name must be derived, validated and advised even though it contributes no entry point or manifest entry. The population whose faults are currently unreportable is exactly the population the Name feature is for. Q8.30 REVERSES the import-only scope the Q8.25 landing assumed. The reasoning recorded: if Name is a useful key for imports it is a useful key full stop, so a named template should resolve by that name at run time too, and the asymmetry was an artifact of the wiring rather than a designed boundary. The manifest carries the name, the runtime registry consults it, and the assertion asked for is of that behaviour -- not, as the question originally proposed, that a runtime name lookup misses. Q8.31 moves the #line path-form choice from prose in generated code into the manifest as machine-readable data, and deletes the comment. Q8.30 and Q8.31 both add manifest fields, so they share one schema bump. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lidation Completes the work checkpointed in d1fa89f. Suite green again: 5414 passed, 0 failed, 0 skipped -- the one red test that WIP commit recorded is reconciled. Q8.30 -- PrecompiledTemplateInfo.RegisteredName, and the runtime resolves by it. The generator emits only a name that actually registered at build time, so both tiers hold the same spellings. PrecompiledTemplates gains a second index and TryGet consults keys then names, so TryResolve, TemplateResolver and PrecompiledRuntime all get name search without their own changes. KEYS WIN, always, independent of registration order. Three grounds: additivity (an addition that displaced an existing spelling IS the override Q8.25 corrected); the match principle (the build-time import map is already keys-first, so the tiers cannot disagree about what a spelling means); and determinism (one merged dictionary would make the winner depend on which assembly loaded first, which is the host's business, not ours). Mechanically that is two indexes plus a disjointness invariant enforced from BOTH directions -- a name whose spelling a key owns is refused at insert, and a key arriving later evicts the name shadowing it. Both directions are needed because either can happen first across assemblies. The cross-assembly collision gets a RUNTIME id, HED7104 (PrecompiledFallbackReason.RegisteredNameUnavailable), because the build tier structurally cannot see it: a referenced manifest's rows live in a GetTemplates method body -- IL, not symbol metadata. It deliberately does not join the duplicate-key throw, since two templates claiming one key is unresolvable whereas a name/key collision is already resolved by the ordering rule. Q8.31 -- LinePathForm on the manifest as an enum rather than a bool, because a fallback-marker row has no generated source and must be able to claim nothing (Unspecified). The enum file is linked into the generator so a rename breaks its compile instead of emitting a manifest that will not compile downstream. The prose comment is gone from generated code, asserted together with the field's presence so an implementation leaving two carriers cannot pass. Q8.28/Q8.29 -- key and name derivation moved ahead of the Precompile gate, so a malformed Key, a malformed Name or a taken Name now reports HED7004 on the item that is actually wrong rather than surfacing as HED7011 at some importer. Opted-out items parse in advisory-only mode so their own imports draw HED7028. HED7029 was not needed: every fault had a home. A "name is build-time-only" warning was declined because it would fire on the feature's intended shape. Opted-out items still contribute no entry point and no manifest entry -- pinned, and verified end to end from the real sample csproj. Schema stays at Min = Max = Current = 3. The 11-arg PrecompiledTemplateInfo ctor is kept as a REAL overload rather than defaulted parameters, deliberately not repeating the mistake that caused the whole schema episode; the public-API diff confirms it survives. Mutation testing: 17 run, 16 killed. M6 was a real gap -- checking names against pre-registration keys survived, because the name landed in the index violating disjointness and the lookup still returned the key owner (two redundant guards masking each other). Fixed by asserting the HED7104 report, not just the resolution. The surviving mutant is proved extensionally equal against the code: one insertion site each, byKey never removes, and pass 2 only inserts spellings absent from the complete staged key set, so at most one dictionary holds any spelling and lookup order cannot matter. Q8.2's breaking-window disposition is REPLACED, not softened -- its ground (a), that the break already shipped in 2.0.0, is withdrawn outright. Q8.30 is recorded honestly as a widening taken anyway: scoped to opt-in data no 2.0 project can have (the metadata was inert until 2.1), nothing that resolved before stops resolving structurally, and the shadowing hazard is reported rather than silent. Five new questions recorded, the sharpest being that RegisteredName has no gauntlet coverage (every other manifest field is validated at request time) and that Min == Max makes the support window a point, so every future manifest change becomes a whole-assembly rejection for the previous version. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two probe compilations graft a tree onto the consumer's own compilation — the `@using` "does this directive compile" check and the embedded-C# expression typer. Both parsed with the defaults. Roslyn requires every tree in a compilation to have been parsed alike and throws `ArgumentException: Inconsistent syntax tree features` on the graft otherwise, so the emitter reported HED7020 and nothing precompiled for any consumer whose project sets a language version, a preprocessor symbol or a compiler feature. That is not an exotic project: a plain `dotnet build` on the current SDK already passes `/features:InterceptorsNamespaces=…`, and it took two shipped samples down — `codegen-t4-successor` and `precompiled-app`, both of which carry a `@using`. Measured by building `samples/codegen-t4-successor` at each commit in the range: clean at ec613c2 and 703567d, broken from 08872ff onward. ec613c2 introduced the typer's graft and 08872ff the directive check's; the samples' templates carry a `@using` and no embedded C#, so only the second is visible there. Both sites break on their own — reverting either one alone reddens exactly its two rows of the new test. Each probe tree now carries the parse options of a tree the compilation already holds, which is what makes the graft legal and is also the only reading that asks the question the way the consumer's own compiler will answer it: its language version decides whether an alias directive binds, its preprocessor symbols decide what an expression contains. The defaults stand in only where the compilation holds no trees at all, where there is nothing to be inconsistent with. The generator takes no parse options from its pipeline, and adding `ParseOptionsProvider` would be a second source that merely coincides with this one. Nothing else in the repository builds a tree it grafts onto a compilation it did not parse: the engine's two C# tier sites and the emitter's metadata probe each construct a fresh compilation, one over its own single tree and one over references only. No suite could see this — every generator harness built its compilation from zero trees or from trees parsed with the defaults. `GeneratorHarness` now takes parse options and hands the same ones to the driver, as the real build's does, and the new test runs both probes under a consumer's options with a same-bytes neighbour under the defaults and a neighbour that must still be refused. Also measured, and left alone: rebuilding the typer's probe as a standalone compilation over the references would close the internals-visibility gap recorded against it, and costs the model type. A model declared in the consumer's source is in no reference at build time, so every expression over it stops compiling and the template silently leaves the precompiled tier. The generator integration suite stays 1017/1017 green under that mutation because its harness seeds no consumer source; the new test is the first thing here that observes it. Suites, Debug and Release, both TFMs where multi-targeted: Heddle.Tests 2043, Heddle.Generator.Tests 586 (582 + 4), Heddle.Generator.IntegrationTests 1017, LanguageServices 200, Tool 6. All 10 samples build and match their goldens. net6.0 was not run: no .NET 6 runtime on this machine. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every regression check in the findings register is a filter command, and a filter naming a test that has been renamed or deleted ran zero tests, printed nothing at quiet verbosity, and exited 0. That is how two shipped fixes came to be documented as pinned with nothing executing them, and how renaming a test in one commit silently disarmed a check written in another. TreatNoTestsAsError makes an empty run fail, which turns all 106 checks from "green if the name still exists" into checks that actually run. The defect this closes had the wrong diagnosis recorded against it. Both instances it cited -- a declared framework with no runtime installed, and a wrong-framework invocation -- exit 1 on this SDK and were never the fault. The filter case is the live one and had never been written down. The first version of the settings file broke every run: an XML comment containing a double dash, which XML forbids, so the file was rejected and real filters exited 1 as well. Caught only because the positive control ran beside the negative one. A guard is not verified until both of its answers are measured. Also records the probe parse-options fix, whose reach was wider than the two samples that exposed it: the SDK passes a features flag on a plain build, so every consumer compilation carries non-default parse options and every consumer with a using directive or an embedded expression was breaking. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Switching on the empty-run guard immediately failed 26 checks -- every one written as Name~. The names were not stale: Name~ with the exact, correct, full method name also matches nothing, because this xUnit adapter supports only FullyQualifiedName. A quarter of the register's checks had never executed anything and had passed every time they were run. Two more named the wrong project. Both came from proposed deltas I merged without verifying where the tests actually live. The consolidation pass claimed to have validated all of these against --list-tests. It did so by reimplementing the filter grammar and matching against a list of names instead of invoking the tool, and its reimplementation assumed Name worked -- the same defect class this register keeps finding in the source, committed by the harness while verifying it. All 74 unique checks now run and pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…by its element Two seams where the emitter held a symbol and passed on a lossy summary of it. The export overload binder was handed the shared OperandKind descriptor, which names the numeric primitives, bool and string and nothing else. A struct, an enum, a class and object therefore all arrived indistinguishable from an argument nothing had resolved — and an unresolved argument is exactly what the sole-candidate shortcut may bind past, since one candidate settles which overload is meant. Three argument categories, measured: a Money member, an OrderStatus member and a Manufacturer member each wrote a call the consumer's compiler answers with CS1503, twice, off a .g.cs nobody can edit, where the engine answers HED1012 before it renders a byte. The binder now takes the symbol the writer already resolved, which is what NativeExpressionCompiler hands OverloadRank on the other tier. That shrinks what "cannot say" covers rather than widening what it may mean; deleting the shortcut instead was measured and rejected, since it sends every single-params export down an expanded tier this writer does not emit. Widening the estimator re-opens the one position that spells a name with no classifier in front of it: the per-argument cast target. It was safe only because of what the estimator could reach, and an argument carrying its own symbol reaches a host's whole signature set, where an [Obsolete(error: true)] parameter type is CS0619 in the consumer's build. It goes through ClassifyTypeName now and degrades where the name is out of reach. An @list body was emitted on the dynamic tier although ListExtension.InitStart gives the engine the collection's IEnumerable<T> argument and the engine compiles the body in a scope of that type. The emitter computed that type — every gate that types a call site inside the body already read it — and then bound the body's own reads dynamically, which cost it in both directions. A member the element type does not carry precompiled and threw RuntimeBinderException at render where the engine refuses at compile; over an element type of object it precompiled and RENDERED a page the engine refuses outright. And a native expression, a function call or a ternary over the element's own member degraded a template the engine renders. The body is typed by the element now, and the element type passes the same name gate every other spelled type passes before it reaches the file. Corpus and samples: zero newly degrading, zero newly precompiling, zero moved bytes — and zero moved generated source, because no corpus template pairs a declared model with an @list, which is why this sat open. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ntegral tiers, and stop two path questions being answered by the wrong grammar
Six deferred engine defects, each closed against the thing that owns the rule
rather than against a recollection of it.
A bool/bool? bitwise pair is refused with the positioned diagnostic every other
illegal operand pair carries. VisitBitwise computes a lifted flag for all three
of its arms and spent it in only two; bool has no promotion to spend it on, so
the mismatch is refused instead. The shared operator table already published
that verdict, so the runtime and the table now agree.
round/floor/ceil carry the int and long tiers abs/min/max already carried. The
register named C# betterness as the fix; a compiled snippet says otherwise —
Math.Floor(3) is CS0121, because double and decimal are mutually
non-convertible and neither is closer. What ties an integral argument is the
overload set, not the ranker, and the ranker still refuses a genuine tie. The
betterness quantification moves with the table it measures: 62 combinations
ambiguous under both schemas become 38, and 82 become 100.
A using alias binds the head before the name index and commits, which is the
order C# reads a namespace-or-type-name in. Established by compiling and
executing the cases, and kept honest by a gate that asks the compiler what it
binds rather than restating a rule. The arms are mirrored on both tiers, and
the deviation this closes is removed from the candidate register.
A hosted GetTemplate can load the file its own search found. The search
inserted views/{controller}/ between root and file name and the options then
dropped exactly that segment. TemplateName means the path relative to RootPath
— which is what the precompiled partial path already assumed — and the search
now hands back the candidate it matched instead of it being re-derived. The
non-hosted arm had the same defect for a caller passing a nested path.
An import path is not a template key. The key grammar strips ~/, strips a
leading /, and appends .heddle, and each of those three names a different file
than Path.Combine does — which is the contract the documentation states and the
engine implements. The build tier now declines to rename a path spelling, and
still honours a spelling a Key or Name registration genuinely renames.
An import-only fragment no longer reds a build over a state no run of the
application can reach. The errors held are the ones an importer could have
satisfied — a base definition the file does not declare — so a syntax error,
which fails the same way in either setting, is unaffected; and they are only
withheld where something else in the compilation imports the file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four of these were parked on a stated reason that measurement refuted. The rounding cure would not have worked -- the same call is CS0121 in C# for the same reason, so the gap was an overload set two tiers short, not a binder. The import spellings the documentation was said to teach belong to a different feature on a different code path, and for imports the docs state the opposite. The fingerprint said to block the hosted-template fix does not contain either field it was said to contain. And one entry had been fixed incidentally eight cycles earlier; establishing that took 233 measured cells. They survived because the review protocol said not to re-report a known-open, which every reviewer read as permission to skip it, so the stated reasons were never tested. The protocol now separates the two, and every remaining known-open carries a runnable check, so a claim that has expired fails like any other check. Two entries understated what they described. The list-element body carried an unrecorded silent-wrong-output cell, because object is a static type rather than the absence of one. The export binder defect had three argument categories, not one, and widening the estimator to fix it re-opened the unchecked cast exactly as this register predicted it would. Three defects found while closing these are recorded as open rather than left in a report: a this argument in a native expression, a missing built-in diagnostic, and a using-namespace exposing nested namespaces that C# does not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…op a using reaching a nested namespace `this` inside a native expression degraded on the precompiled tier where the engine renders it. The expression writer had an arm for every node shape but that one, and a `this.`-rooted hop was refused along with every other targeted path. It now emits the model local — which is what the engine's compiler converts the model parameter to — and roots a member walk at the model, skipping the prop layout exactly as the engine does for a path with a target. A body with no static model still degrades, and `this` standing alone stays the model passthrough it always was, so the two constructs keep their separate rules. The arm was added everywhere the writer and the emitter dispatch on node shape: emission, the operand estimate, the argument's resolved type, the path root, and the call-site value typing. The built-in overload binder was fed only the shared operand descriptor, which calls an `object` argument "cannot say" on purpose because an object-typed operand carries no facts an operator can key on. That is not the same as having no type, and `min(1, Payload)` degraded in silence where the engine refuses it by name. The binder now takes the writer's resolved symbol beside the descriptor and names `System.Object` exactly — the one type the descriptor drops that a name-keyed rank model can spell, and one the whole shipped candidate set decides without the reference conversion that model must answer false to. Two functions that accept the same argument and used to degrade with it, `str` and `format`, pre-compile again. The four other things the descriptor calls unknown — dynamic, an error type, a type parameter, no symbol at all — keep the silent degrade they had. A `using` namespace import exposed the namespaces nested inside it, which C# does not: `using A;` plus `Sb.Deep` resolved here and is CS0246 there. Both tiers agreed with each other, so no differential could see it; the rule was established by compiling and executing the spellings and then followed on both tiers. A candidate counts only when its own namespace is the import, which keeps a nested type reachable — it reports its outer type's namespace — and a nested namespace not. This takes working templates off both tiers and is recorded as the narrowing it is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
xUnit drops a theory row whose arguments duplicate another row's, logs "Skipping test case with duplicate ID", and leaves the run green at a lower count -- Skipped: still reads 0, because the row is dropped rather than skipped. The empty-run guard cannot see it: the filter still matches tests, so the run is not empty. The log line is the only signal, so CI now fails on it. An agent introduced such a row this cycle and caught it only by reading a Release log line by line. Of the three defects recorded here, one was two constructs sharing a spelling: this as a whole call parameter is the model passthrough and needed no static type, which is why that path worked and this as an operand had no arm at all. A golden had pinned the empty manifest that resulted, under a name promising the call it was missing. The built-in binder defect was wider than recorded -- two built-ins degraded where the engine renders, not only the one that lost a diagnostic. The narrowing to match C# on nested namespaces cost 0 templates, fixtures and samples, and 1 test whose spelling moved to a nested type so it keeps making the same claim. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The sample metadata entry claimed the build no longer reads Name. It does: the generator pulls it, uses it for the import advisory, and the props file declares it visible with its contract stated alongside. The entry also describes a template and a metadata pair the sample does not contain, and the sample builds and matches its golden. Overturned. The type-kind entry is the first parked reason in this sweep that survived being tested. Structure is a compile-time alias of Struct so a second case will not compile, and Extension is not declared by the Roslyn the generator references; raising that reference raises the minimum SDK every consumer needs, which is a cost rather than a defect. Its check now names the version to watch, so the Extension half is re-tested if it ever moves. That closes the sweep: of the sixteen known-opens standing when it began, six were parked on a reason measurement refuted, one had been fixed eight cycles earlier, and one more was fixed without the entry being updated. Six remain, four of them blocked by this machine rather than by choice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… what closing it needed The recorded residual said nothing drove a boxed non-string through the value path on both tiers, and that closing it needed a host extension whose ProcessData consumes its body's Execute result. The first half was already stale — strategy-nonstring-value-bytes and the three-case theory byte-compare the drop across tiers — and both redden with the rail mutated on either tier, in Debug and Release, so they are load-bearing. The second half is not buildable. AbstractExtension hard-codes typeof(string) as the declared return of every extension that leaves InitStart alone, so the engine faults on the returned type before its coercion runs; declaring anything else means overriding InitStart, which is HED7015 for a roleless non-engine extension; and a bodied custom call degrades unconditionally. Engine-assembly extensions are the only vehicle. New ValueRailCoercionDifferentialTests carries the host-extension half as far as it goes, and pins the bound rather than working around it. The return-type fault is #if DEBUG and the emitted Execute has no counterpart, so the tier comparison is the Release leg and DEBUG pins the fault; the bodied degrade is paired with the bodiless neighbour, which still precompiles and renders. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ostic ids Eleven id-carrying warnings were added to CompileWarnings and reached no build-time diagnostic. The recorded gap assumed closing it meant running compile-channel stages in the generator; it did not. Ten of the eleven conditions were already decided by the generator's own walk, and the eleventh by a predicate it already supplied to the shaping pass. The shaping-time warnings move into shared cores that both compilers drive, rather than being decided twice. BranchSetLint is the observer StripBranchSets always accepted and the generator always passed null for (HED3001, HED3002, HED3004, HED3005); OutputLints holds the two scans that read nothing but text and parse spans (HED4005, HED2004). Both were lifted out of HeddleCompiler, which now drives them. The remaining five are decided per tier — reflection on one side, Roslyn symbols on the other — but their text lives once, in CompileWarningFactory, so a build report and a run report of the same mistake are the same sentence. Mutating a sentence there reddens the runtime's own test while the cross-tier parity test stays green, which is the demonstration that there is one source and not two. Three stay narrower at build time and say so: HED1016 sees the default function table and each assembly's exports but not a registry the host fills at run time; HED5011 answers for symbols the compilation can see; HED2003 reads the attribute off a bound extension type. Narrower is the degrade direction, and NothingIsForwardedThatTheRunTierWouldNotRaise gates the other one. The corpus gains BuildForwarded beside ParseChannel — they stopped being the same set — plus five fixtures, measured on all three hosts. The parity arm compares both tiers' live output for the same bytes instead of restating either. Every arm was rehearsed red: ten mutations, each reddening only its own id. Measuring the error arm of the same event stream turned up a live divergence that is not fixed here: an orphan branch terminal is a compile error the engine refuses the template for, and the generator precompiles it and reports nothing. Draining that arm fails a consumer build on the generator's own classification, which is a different blast radius from eleven warnings. It is checked in as a skipped red test with either half of a fix as its acceptance. Also corrects three claims in the README that no longer held: two of the three "shared cores with no runtime caller" have runtime callers and no surviving second copy, and the mutation sentence that stood as their evidence is false in both directions when measured; the generator has two blanket catch sites, not three; the catalog is 91 rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… a refusal Two faults that both ended with a tier telling the caller something the engine would not have said. A model of the wrong type reached the compiled accessor's cast and escaped as a raw InvalidCastException, which is not the shape any other render fault has. The top-level render now checks the model against the scope type once per render -- outside the debug-only guard, and not per processor, so the recursive path is untouched. The precompiled adapter has no compile-time model type to check against and is skipped there, as it always has been. The shared branch scan already knew when a body contained a terminal with no opener; only its warning arm was wired up, so the build tier precompiled bytes the engine refuses to render. Its error arm now reaches the emitter, which declines the body and leaves the template to the dynamic tier. branch-import-else.heddle is the corpus entry that proves the second one: a bare @else continuation, previously declared as precompiling, now falls back. It was also the only row that was both precompiling and excused from rendering, so the theory reading that column lost its last case. xUnit errors rather than passing on an empty MemberData, so the check is now a fact with the cardinality asserted ahead of the loop -- the emptiness is the claim, and a new row reddens it. TemplateOptions.ValidateModelType is left in place but no longer read; removing it would be a break. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tributed ones on DCO
Three jobs built and shipped in one step, so a publish could begin from a tree
whose build had not finished proving itself, and the credentials to publish were
present for the whole build. Each is now a pack/build job that ends in an
artifact, and a deploy job that starts from one. Only the deploy half holds
publishing permissions.
dotnet: prerelease -> prerelease-pack + prerelease-publish
release -> release-pack + release-publish
npm: publish -> build + stage-prerelease + publish-release
The DCO workflow becomes callable as well as a required PR check, and the two
publishes that ship contributed code depend on it: an unsigned commit stops the
beta rather than staging it. Releases from main are deliberately not gated -- a
tag push carries no pull request to check, and they are the maintainer's own.
docs already had the split and deploys only from main, so it is unchanged.
The lsp and samples workflows upload artifacts rather than deploying.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ed row Two checks that could not fail. The resolver sweep's file-backed half asserted only inside a loop over a set it derived itself, so emptying that set left it green having rendered nothing. Its twin, one method above, caught the same mutation on the first line of its body -- the difference between them was the count. Both now assert the swept set's size, which is what the loop iterates; the existing count pinned the targets going in, a weaker claim than the one the loop makes. The duplicate-row guard wrapped the Debug run alone, so the two Release legs could lose a theory row with nothing to notice. It was also bash inside a step that runs under pwsh on the windows leg, where it would have failed rather than guarded -- latent, because these workflows only trigger on main and this branch has never run them. The guard is now one script behind every dotnet test call in CI, including the two in the LSP workflow that had none, and both workflows pin the shell instead of inheriting the runner's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
.NET 6 is out of support and was the floor holding back everything else, so it goes from every target framework, package group and conditional in the tree. The two shipping libraries drop to netstandard2.0;net8.0;net10.0, the test project to net8.0;net10.0 plus net48 on Windows, and the benchmark project to net8.0;net10.0. Its net6-only debris goes with it: the ASP.NET Core 6.0.36 group, and the BenchmarkDotNet version split that existed only because 0.15.1+ had dropped net6.0, which collapses back to one unconditional reference. 41 `#if !NET6_0` guards across 18 benchmark files are now vacuous and removed -- four times what the first survey of them reported, so they were counted from source rather than sampled. Each was checked for an `#else` branch first; none had one. The `#if NET6_0_OR_GREATER` in the sink benchmark goes too, being always true once the project floors at net8.0. The 17 other NET6_0_OR_GREATER guards stay: they sit in projects that still target netstandard2.0 or net48, where the symbol is genuinely undefined and the guard still selects behaviour. The two package pins net48 consumers resolve transitively move from 6.x to 8.x -- FileProviders.Embedded to 8.0.29 and Encodings.Web to 8.0.0 -- both confirmed from their nuspecs to carry the netstandard2.0 asset, and Encodings.Web a net462 one, which is what net48 actually binds. CheckEolTargetFramework stays but its comment no longer claims net6.0 is being kept deliberately; it now records the real reason, which is that net8.0's own LTS window closes in November. Suites unchanged at 2134/606/1115/205/6 in Debug and Release, 0 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The sweep that removed the vacuous net6 guards only looked for `#if !NET6_0` and `#if NET6_0_OR_GREATER`; it never checked for a bare `#if NET6_0`. There are none -- confirmed over every source file in the tree, not just the benchmark project the earlier survey covered -- so the 41 removed were the whole set. What remained were 16 NET6_0_OR_GREATER guards in 12 files, all still live: they sit in projects that also target netstandard2.0 or net48, where the symbol is undefined and the fallback arm is the one that compiles. Live but misleading, since they name a framework this repo no longer supports while actually meaning "not netstandard2.0, not net48". With the floor at net8.0 the two symbols select identical TFM sets -- false on netstandard2.0 and net48, true on net8.0 and net10.0 -- so the rename preserves behaviour exactly. The netstandard2.0 leg was built on its own, since that is the leg whose fallback arm the guards exist for and a whole-solution build would not distinguish it. Suites unchanged at 2134/606/1115/205/6 in Debug and Release, 0 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…h context Two tests grew a System.Runtime.Loader dependency in late July and broke the net48 leg, which is Windows-only and had not run since. .NET Framework has no AssemblyLoadContext at all; the engine's own AssemblyHelper already guards its ALC arm for the same reason. The using, both facts and the shared helper go behind `#if !NETFRAMEWORK` -- the helper in the same guard as its only caller. This is the file-level convention already in use: NetFrameworkDegradePathTests wraps its net48-only fact in the positive form of the same symbol, and no csproj in this repo conditions a Compile item. Each guarded test says in its doc comment that the engine rule it pins is not framework-specific; only the probe's mechanism is. Verified by compiling the net8.0 leg with NETFRAMEWORK forced on, which is the branch net48 takes and the only way to reach it from Linux. That alone would be a weak check -- the ALC type still exists on net8.0, so an unguarded use could compile regardless -- so it was rehearsed the other way: removing the helper's guard fails with CS0246 on AssemblyLoadContext, the same root cause CI reported as CS0234 on the namespace. The first attempt at that rehearsal reddened on an orphaned #endif instead, which proved nothing and was redone. Heddle.Tests unchanged at 2134 in Debug and Release. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every test project becomes a stand-alone executable hosting its own runner: xunit 2.9.3 -> xunit.v3 3.2.2, Verify.Xunit -> Verify.XunitV3 31.28.0, and the VSTest host and adapter drop out entirely. Verify.XunitV3 pins the same 3.2.2 extensibility package, and xunit.v3 ships net472 alongside net8.0, so the net48 leg keeps its runner. The cutover had to be all five suites at once: global.json selects the runner for the whole repo and refuses a mix, which one suite proved before the rest were touched. That smoke test also caught the worst failure mode first -- removing the VSTest adapter without selecting MTP leaves `dotnet test` printing nothing and exiting 0. Three code changes, which is all v3 asked for: the one Xunit.Abstractions import, IAsyncLifetime returning ValueTask, and reading InlineDataAttribute.Data rather than a GetData overload that is now asynchronous. tests.runsettings is deleted. Its TreatNoTestsAsError is native here and stronger: --minimum-expected-tests exits 9 below a floor, and a filter matching nothing exits 8 rather than 0, which retires a way checks have died in this repo. The CI wrapper stops scraping the log for a v2 message that no longer exists and passes each suite's real size instead; every suite now has its own named leg, since one aggregate floor cannot see a single suite going quiet. All three arms of the wrapper were measured, not assumed. Randomised test order immediately found a genuine order dependency: two tests built probe assemblies whose names were unique but whose model type was not, so whichever ran first decided whether the engine could see the type. Both now carry a per-probe namespace, which is what "not loaded yet" needed to be a fact. It also exposed a real tier divergence, checked in skipped: an assembly-qualified model version ahead of the assembly's own is ignored at build time and refused by the engine, so the same template precompiles and throws. The old host satisfied the request from the loaded assembly whatever version was asked for, which hid it. Left for its own change rather than folded into a runner move. 2134/606/1115/205/6 in Debug and Release, one deliberate skip. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ys there Zero in Debug and Release, across the solution and the twelve projects it does not contain -- the samples, the dotnet benchmark harness and the ThirdParty comparison project, none of which a solution build reports on at all. The v3 analyzers found real assertion defects, and those are fixed rather than suppressed: Assert.Single and Assert.Empty over a Where clause both discard the predicate from the failure message, so a red row said only that a count was wrong and not which element was missing. Same for a collection size compared with Assert.Equal. An obsolete model type had a pragma covering its construction but not the typeof one line below, which is the same deliberate use. Two package references are deleted rather than pinned: System.Net.Http has no usage anywhere in the tests, and System.Text.RegularExpressions is in the box on every framework still targeted, net48 included. Three suppressions are kept, each with its reason next to it. A concurrency test joins its writer thread synchronously because the assertions are meaningless until it stops. The Razor harness uses runtime compilation because that is the mode it exists to benchmark. Vector tables carry a row label their bodies never read, whose job is naming the failing row out of dozens. The largest rule, xUnit1051, is turned off per test project with its reasoning: it wants a cancellation token threaded through 46 synchronous Roslyn calls in a suite whose tests finish in milliseconds, so there is no hang for a token to shorten. Recorded as a decision, revisited when a cancellable test appears. The practice itself now lives in the coding standards: sweep per change while the context is fresh, build stays permissive on purpose so a warning never blocks running the tests, sweep both configurations and the projects outside the solution, and record anything deliberately kept instead of silencing it. An XML comment written with a double hyphen broke all four project files at once -- the same way it broke the run settings earlier in this work -- so the csproj files are now parsed as XML before the build is trusted. 2134/606/1115/205/6 in Debug and Release, one deliberate skip. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each was written as a test and run before being believed. Three outcomes from four reports, which is the reason for measuring rather than transcribing. Confirmed, checked in skipped. A type nested in a generic outer: the engine counts GetGenericArguments().Length, which includes the outer's parameters, and the build tier counts Arity, which does not, so a spelling the engine binds is silently left on the dynamic tier. A dotted spelling reachable through two imports: both tiers let @using declaration order pick a winner where C# raises CS0104 and refuses the spelling outright. Overturned, and kept open and green. The keyword table really is consulted at opposite ends of the two walks -- the engine's alias arm claims the head before its table, the build tier reads its table before the index carrying aliases -- but the difference is unreachable. It needs an alias whose name is a keyword, and C# permits that only for `dynamic`, every other name in the table being reserved. One name is the whole population and both tiers agree on it, so the row stays as the pin that keeps it that way. The first version of that row proved less than it looked: equal bytes are also what a build-time degrade produces, so it now demands the template precompile. The arity row was wrong too, spelled with a literal backtick the grammar builds for itself, and failed on the engine before reaching the arm it names. The fourth report, an import-versus-static CS0104 bucket recorded as both done and not done, has one mention left in the docs and no contradiction to fix; the behaviour it described is the dotted-import row above. Both spec documents gain the missing branch. A finding becomes a test whichever way it lands: red and reproducing is skipped with its owner, green because the claim was wrong stays open as a regression pin, and green where the intended behaviour is unclear is a question for the maintainer, not a guess to assert. 1119 in the integration suite, 3 skipped, zero warnings, Debug and Release. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t replaces Maintainer ruling: src/Heddle.Performance is decommissioned for comparative benchmarking. The .NET leg of the cross-stack program is rebuilt under benchmarks/dotnet/, structured like the other five ecosystem harnesses, and nothing in the tree references the old project's code. Recorded as ledger E12. Why a rebuild and not a move. The old project was a SIGNED src/ assembly and a friend of the engine, so it could and did reach internals no caller can -- and a benchmark that needs private access to the thing it measures is measuring something no user can reach. It also multi-targeted three frameworks for an intra-.NET historical question this program does not ask, stored its templates as escaped C# literals rather than reviewable files, measured one fairness track, and covered Razor on one workload of eight. ## The harness (W8-W12 of 12; W1-W7 landed in 02405ba) One unsigned net10.0 project, verb-dispatching like every other harness: gate 143 cells: byte gate, verifier, security floor, plus the materialisation trailer and a stated precompiled coverage selftest 141 checks incl. the six-technique differential export-corpus regenerate from the Heddle oracles verify-corpus freshness + verifier calibration bench-crossstack 8 workload-named suites x 6 engines x BOTH fairness tracks bench-techniques Heddle's six render techniques against each other bench-cold cold parse/compile, per engine bench-internal props, branching, language-service metadata Bench verbs forward their remaining arguments to BenchmarkDotNet, so a master runner selects one suite per step and layers the measurement budget on top. The corpus is what proves the replacement faithful rather than merely new: re-exporting reproduces all sixteen committed artifacts -- eight goldens and eight .verify.json sidecars -- byte for byte. Three things are new rather than ported. The default job moved out of a [ShortRunJob] attribute into configuration: an attribute job cannot be REPLACED from the command line, only added to, so `--job Dry` used to run the full measurement as well as the dry one. A filter matching nothing now exits 1 instead of reporting a green step over zero rows. And the internal suites are measured per operation rather than over a 10,000-iteration loop, which only multiplied the number it was meant to expose. The internal suites' models are TOP-LEVEL types. A nested type's reflection name carries a '+' that the definition `::` clause cannot lex, so a suite whose models were nested inside it would fail to compile its own templates. ## The decommissioning The project is deleted. With it go the engine's InternalsVisibleTo grant, the solution entry, the .gitignore line, and Directory.Build.targets' CS8002 acceptance mechanism -- which existed solely because one signed project under src/ referenced unsigned Scriban, and no project under src/ references a competitor engine any more. VersionConsistencyTests keeps the property that mechanism protected: no project buys silence with a project-wide NoWarn. The vendored third-party Fluid harness inside it is PRESERVED, not deleted: it is an independent deliverable with its own upstream obligation. It moves intact to benchmarks/third-party/, renamespaced, and still passes its parity check. Both master runners, the Phase 8 launcher and consolidate.py are cut over. consolidate.py maps artifacts by workload-named suite class, reads the Track column so the idiomatic tables finally have a .NET row to anchor to, and filters the ratio baseline to the controlled track -- an unfiltered comprehension would have let whichever track appeared last become the baseline every ratio divides by. Two consequences worth stating rather than discovering: Phase 1 D15's controlled-track-only scope for .NET is DISCHARGED, and the .NET measure phase roughly doubles, so ledger E6's ~9/~21 min figures for that leg are now low. ## Documentation DocumentationLinkTests enforces link integrity, so leaving the ratified specs' citations dangling was not an option: every link across 31 documents is repointed at the replacement file. The surrounding prose stands as ratified and still names the old project where it records what was researched at authoring time; the full mapping is ledger E12, with banners in the plan and spec indexes pointing at it. docs/plan/README.md is restated end to end -- the leg is complete, and measurement is the only work left. ## Verified Solution and both harnesses build clean, 0 warnings, Debug and Release. gate 143/143, selftest 141/141, verify-corpus verified. run-all.ps1 -Smoke -Ecosystem dotnet: 15/15 steps green, gates through artifact copy. consolidate.py against that run: 96 cross-stack cells, both tracks, eight workloads, six cold cells, 49 internal rows, anchor complete. Tests 10,027 passed / 22 failed -- down from 25, and every remaining failure is a pre-existing Windows or net48 issue untouched by this work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…pled
Ledger E6 set "one uniform measurement budget of ~10 min per ecosystem", and
every leg it was sized against carries two engines -- Rust, JVM, JS, Python --
or three (Go). For those legs the leg total and the per-engine share are nearly
the same number, so the distinction never surfaced. The .NET leg carries SIX:
Heddle, Fluid, Scriban, DotLiquid, Handlebars.Net and Razor. Holding it to the
same LEG figure divides one ecosystem's budget across three times as many
engines, so every .NET cell is sampled about a third as well as a Rust cell --
and the cross-engine comparison this program exists to make would rest on its
worst-sampled rows.
The budget unit becomes ONE ENGINE. short and baseline keep their ~10 and ~30
minute sizes; what they size is an engine's share. A leg's total is its engine
count times that, so the .NET leg is longer than the others by construction
rather than by accident.
## Launches, not iterations
Measured on this repo, one 12-cell suite, quiet machine:
LaunchCount 1 139 s 11.6 s/cell (BenchmarkDotNet's stock ShortRun)
LaunchCount 2 316 s 26.3 s/cell (the new committed short shape)
LaunchCount 5 704 s 58.7 s/cell
About 10.8 s per cell per additional launch across L2..L5 -- very nearly linear,
which is what makes the knob predictable. Raising the iteration counts instead
runs through BenchmarkDotNet's pilot stage and does not: a trial at
--launchCount 4 --warmupCount 5 --iterationCount 8 overshot its projection by
more than 2x and was abandoned.
That is the practical reason. The substantive one is that LaunchCount 1 samples
the within-process term ONLY -- one process, one JIT, one heap layout -- and
never samples the cross-process term at all. E6 itself established this is the
term worth paying for: it kept JMH's forks plural after measuring fork-to-fork
RSD at 1.18% median against a within-fork 0.18%, and re-spent the whole JS
budget on independent processes for the same reason. The .NET leg was the one
that had never bought it.
short ShortRun + LaunchCount 2 66 min leg 11 min/engine (measured)
baseline --launchCount 6 ~175 min leg ~29 min/engine (projected,
one launch past the last measured point -- replace it with the real
duration after the first protocol run)
The leg is 151 cells: 96 cross-stack (8 workloads x 6 engines x 2 tracks) plus
the techniques, cold and internal sidebars.
Verified after the change that the committed default really is LaunchCount 2 and
that --launchCount 6 MUTATES that job rather than adding a second one -- the
report CSV carries two rows at L6, not four.
## Recorded and deliberately not acted on
The other five legs are still sized per ecosystem, which leaves them at roughly
3-5 min per engine against .NET's 11. Making the program uniform per engine
means roughly doubling every one of them, and that is a session-length decision
for the maintainer rather than a consequence of this amendment.
Filed as ledger E13, which amends E6's unit and its .NET row, the metrics
protocol's .NET harness row, and the benchmarks/README budget tables.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…hmetic E13 changed the budget's unit from one ecosystem to one engine, resized the .NET leg, and recorded -- without acting on it -- that the other five legs were still sized per ecosystem and therefore sat at roughly 3-5 min per engine. This acts on it: every engine in the program now gets ~10 min at short and ~30 min at baseline, wherever it lives. ## An engine is 16 cells, and that corrects E13 Eight workloads times two fairness tracks. Fixing the definition is what makes the budget checkable rather than a matter of taste -- and it shows E13 got its own .NET figure wrong. E13 divided the whole 151-cell leg by six engines and reported 11 min per engine. That over-credits: 55 of those cells are the Heddle-only techniques, cold and internal sidebars, which belong to no engine's comparison share. The comparable rows are the 96 cross-stack cells, so the real figure at E13's LaunchCount 2 was 7.0 min per engine -- under budget, not over. LaunchCount 3 puts it at ~9.9 min and LaunchCount 10 at ~30.1 min; the sidebars ride at the same job shape on top of that. ## The knobs, all six .NET ShortRun + LaunchCount 3 --launchCount 10 Rust warm-up 5 s, measure 26 s --warm-up-time 10 --measurement-time 84 JVM @fork(5), @measurement(5x1s) -f 5 -wi 2 -i 17 JS 38 aggregated passes 114 passes Python --values 6 --warmups 1 --values 20 --warmups 2 Go -count 28 --count 84 Each ecosystem's committed source or script default remains its short shape, so a bare single-harness invocation is still the budgeted regime. Where a leg had a choice about where to spend the increment it went to the term its own variance decomposition says is dominant: JVM buys forks rather than iterations, .NET buys launches for the same reason (E13), JS buys independent processes (E6). ## Cost, stated rather than discovered Session totals become ~3.2 h short and ~9.7 h baseline, against ~1 h and ~2.5 h under E6's per-ecosystem unit and 5.45 h before E6. That is what sampling seventeen engines' worth of comparison rows equally costs, instead of sampling six of them a third as well as the rest. EVERY duration in the resized table is PROJECTED -- E6's measured per-leg durations scaled by the knob change -- with the single exception of .NET's per-cell cost, measured here at 139 s / 316 s / 704 s for one 12-cell suite at LaunchCount 1, 2 and 5. The first protocol run replaces the whole column with real durations, and the projections should be checked against it before any of these numbers is quoted. Verified that the new .NET default really is LaunchCount 3, that the gate is still 143/143, and that every documentation link still resolves. Filed as ledger E14. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Related issue
How was this tested?
Checklist
dotnet test src/Heddle.Testspasses locally..editorconfig/ surrounding code style.git commit -s).AI-assistance disclosure (if any)