fix(codegen): teach the in-process native backend today's RS4GC IR; unfreeze the dialect corpora - #7998
Merged
Merged
Conversation
…efresh the frozen corpora
`PERRY_LLVM_INPROCESS=native` could not build any function containing an
RS4GC root slot — which is effectively every function. Every `main`
execution of `llvm-inprocess`'s native-backend job failed at
in line: %r5 = alloca ptr addrspace(1)
The issue warned this was not a one-liner, and it was not: fixing each
shape only exposed the next. The family, in the order it surfaced:
1. `basic_type` had no `addrspace(N)` arm — the reported failure.
2. `ty_and_val` split `ptr addrspace(1) null` into type `ptr` and value
`addrspace(1) null`. The qualifier belongs to the type but sits after
a space, so OPERAND position needed the same treatment as type
position.
3. `constant` returned an addrspace(0) `null` regardless of the operand
type, which the verifier rejects when stored into an
`alloca ptr addrspace(1)`.
4. Define lines carry `"frame-pointer"="non-leaf"` (a string attribute)
and `gc "statepoint-example"` (which contains a space, so the
whitespace-split attribute loop saw two junk tokens).
5. Callsites carry `"gc-leaf-function"`.
6. Landing pads are now `landingpad token cleanup`. `token` is not an
inkwell `BasicType`, so that pad is built through llvm-sys. The
`{ptr, i32} catch` shape still occurs and keeps its branch.
Two further defects were hiding behind the reader failure, both worse
than it:
**The native path returned assembly and called it an object.** The
statepoint backends compact the stack map at assembly time, so the plan
carries `-S`. The textual in-process path has always run the
rewrite-and-assemble step afterwards; the native and diff paths returned
the bytes straight to the object cache, and the link died with
`ld: unknown file type`. Factored into `linker::finish_native_emission`
and wired into all four native/diff emit sites.
**A natively-constructed module had NO GC strategy, so RS4GC never ran on
it.** `native_emit::synth_define_header` was a second, independent copy of
`LlFunction::to_ir`'s header renderer, written before
`"frame-pointer"="non-leaf"` and `gc "statepoint-example"` existed and
never updated. The result verifies, links and executes correctly on any
program that does not collect, while having no precise roots at all —
#7332's shape, and invisible to a behaviour-parity smoke arm by
construction. The two callers now share `LlFunction::define_header`, so
the next attribute reaches both; `function.rs` (compiled without the
`llvm-inprocess` feature, i.e. visible to per-PR CI) pins `to_ir`'s first
line against it.
**The corpora, which is the half worth more than the fix.** All three
tracked `.ll` files froze on 2026-08-03, 151 codegen commits earlier, and
contained zero `addrspace(1)`. `corpus_spike ... ok` proved the tests ran,
not that they test today's IR — CLAUDE.md's fourth way a gate cannot fail,
inside the liveness assert written to prevent it. The same thing had
happened nine days earlier (#7310, stale setjmp calls), which is the
argument against fixing it by hand again:
* `scripts/refresh_llvm_inprocess_corpora.sh` regenerates all three from
the built compiler (`--check` diffs instead of writing).
* `scripts/check_llvm_corpus_currency.py` (added to `lint`) asserts every
IR form the reader carries a dedicated branch for is PRESENT in the
corpora, so a form that disappears fails and must be either refreshed
or deleted per the kill policy. Sabotage-verified against the corpora
this commit replaces: it names all 10 forms they lacked.
* `llvm-inprocess.yml`'s existing age diagnostic was itself vacuous —
`git log` on a depth-1 checkout printed `0` commits behind however
stale the files were. Now `fetch-depth: 0`, and it fails loudly rather
than printing a reassuring zero if history is ever missing again.
Corpora refreshed: 497 / 1082 / 420 `addrspace(1)` sites, EH corpus keeps
84 invoke edges and its personality clause. `dialect/mod.rs` crossed the
2000-line cap, so type/constant parsing moved to `dialect/types.rs`.
Validated locally against LLVM 22.1.4: 934 `perry-codegen` lib tests green
(all three corpus round-trips included); `PERRY_LLVM_INPROCESS=native`
compiles and runs the spike and the try/catch program with output
byte-identical to the textual backend.
Known remaining, NOT closed here and separate defects: `=diff` still
reports a byte mismatch on the spike (149,105 text vs 163,902 native — it
was 50,995 before the GC-strategy fix, i.e. the gap narrowed from "RS4GC
never ran" to a real but much smaller divergence), and the unit-split diff
arm fails with `call to undeclared @js_shadow_slot_set`. Neither was ever
reached in CI, because the native arm failed first.
Refs #7982.
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (15)
📝 WalkthroughWalkthroughThe PR expands LLVM IR parsing for address spaces, attributes, constants, GC metadata, and cleanup landing pads. It unifies native function headers and assembly finalization. It adds corpus refresh and currency tooling, CI checks, and related documentation. ChangesLLVM backend updates
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant LLVMIR
participant DialectReader
participant LlFunction
participant NativeEmitter
participant Linker
participant Clang
LLVMIR->>DialectReader: parse supported IR forms
DialectReader->>LlFunction: apply GC metadata and attributes
NativeEmitter->>LlFunction: render function header
NativeEmitter->>Linker: finalize emitted bytes
Linker->>Clang: assemble output when -S is enabled
Clang-->>NativeEmitter: return object bytes
Possibly related issues
Possibly related PRs
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
proggeramlug
marked this pull request as ready for review
August 12, 2026 21:59
The knob-parse audit inventory (every GC-family env knob, how it parsed before and after), the corpus-currency findings, and the #7737 triage — including two things worth more than either fix: * `llvm-inprocess.yml`'s corpus-age diagnostic was itself vacuous. It asks `git log` how many IR-affecting commits landed since the corpora changed, on a depth-1 checkout — one commit to answer from, so it printed a confident 0 however stale the files were. * #7737 item 4's stated prerequisite has gone stale in the opposite direction: `gc-ratchet` has zero successes in its last 30 `main` runs and `gc-stress` failed the last three, so promoting either to required today would block every open PR.
Contributor
Author
|
Follow-up commit
Sabotage-verified with the fix committed first, then restored and rebuilt. 935 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #7982.
The reported defect, and the family behind it
PERRY_LLVM_INPROCESS=nativecould not build any function containing an RS4GC root slot — effectively every function. The issue warned this was not a one-liner, and it was not; each fix exposed the next shape:alloca ptr addrspace(1)basic_typehad noaddrspace(N)arm (the reported failure)store ptr addrspace(1) null, ptr %rty_and_valsplit it into typeptr+ valueaddrspace(1) null. The qualifier belongs to the type but sits after a space, so operand position needed the same treatment as type positionnullin an addrspace operandconstantalways returned an addrspace(0)null; the verifier rejects that stored into analloca ptr addrspace(1)define … "frame-pointer"="non-leaf" gc "statepoint-example"gc "…"contains a space, so the whitespace-split attribute loop saw two junk tokenscall … ) "gc-leaf-function"landingpad token cleanup{ptr, i32} catchwas supported.tokenis not an inkwellBasicType, so this pad is built through llvm-sys; the older shape keeps its branch, both occur in one module todayTwo defects that were hiding behind it — both worse
The native path returned assembly and called it an object. The statepoint backends compact the stack map at assembly time, so the plan carries
-S. The textual in-process path has always run the rewrite-and-assemble step; the native and diff paths returned the bytes straight to the object cache and the link died withld: unknown file type.linker.rseven carries a comment saying exactly that would happen — for the path that already handled it. Factored intolinker::finish_native_emission, wired into all four emit sites.A natively-constructed module had NO GC strategy, so RS4GC never ran on it.
native_emit::synth_define_headerwas a second, independent copy ofLlFunction::to_ir's header renderer, written before"frame-pointer"="non-leaf"andgc "statepoint-example"existed and never updated. The result verifies, links and runs correctly on any program that does not collect — while having no precise roots at all. That is #7332's shape, and it is invisible to a behaviour-parity smoke arm by construction: identical output on a non-collecting program is precisely what it produces. Its only symptom was the diff arm's byte mismatch (149,105 text vs 50,995 native), and the diff arm was never reached because the native arm failed first.Fixed structurally rather than by testing for agreement: one renderer,
LlFunction::define_header. A copy that can drift eventually does, and this one took two attributes and an unknown number of releases to notice. The pin lives infunction.rs, which compiles without thellvm-inprocessfeature, so it runs in per-PRcargo-testrather than only in the feature job.The corpora — arguably the more important half
All three tracked
.llfiles froze on 2026-08-03, 151 codegen commits earlier, with zeroaddrspace(1).corpus_spike ... okproved the tests ran, not that they test today's IR. And it had already happened once, nine days before (#7310, stalesetjmpcalls) — which is the argument against refreshing by hand a third time.scripts/refresh_llvm_inprocess_corpora.shregenerates all three from the built compiler;--checkdiffs instead of writing.scripts/check_llvm_corpus_currency.py(added tolint) asserts every IR form the reader carries a dedicated branch for is present in the corpora. A form that disappears fails, and the fix is a refresh or deleting the now-untested branch — the kill policy applied to fixtures.Sabotage-verified against the corpora this PR replaces: pointed at the
origin/mainfiles it names all 10 forms they lacked, then passes on the refreshed ones. Its--self-testrejects the 2026-08-03 corpus shape and accepts today's.Stated limit, because it matters: this catches a form that vanishes, not one codegen newly invents — nothing static can. The honest closure for that direction is the end-to-end
nativearm, which compiles today's IR by construction.llvm-inprocess.yml's age diagnostic was itself vacuous.git log -- <path>on the default depth-1 checkout has one commit to answer from, so "IR-affecting commits since then" printed a confident0however stale the files were — a diagnostic reassuring readers about the exact thing it existed to detect. Nowfetch-depth: 0, and it fails loudly rather than printing zero if history is ever missing again.Validation
perry-codegenlib tests green under--features llvm-inprocess(all three corpus round-trips included), LLVM 22.1.4.PERRY_LLVM_INPROCESS=nativecompiles and runsspike.tsandtest_gap_7302_invoke_eh_paths.tswith output byte-identical to the textual backend, banner asserted.addrspace(1)sites; the EH corpus keeps 84 invoke edges and its personality clause.dialect/mod.rscrossed the 2000-line cap, so type/constant parsing moved todialect/types.rs.Not closed here (separate defects, neither ever reached in CI)
=diffstill reports a byte mismatch on the spike: 149,105 text vs 163,902 native. It was 50,995 before the GC-strategy fix, i.e. the gap narrowed from "RS4GC never ran on the native arm" to a real but far smaller divergence. The pre-opt prints differ by design (the C-API builder constant-folds at construction), so this needs its own investigation.call to undeclared @js_shadow_slot_set— the per-unit skeleton does not declare it.Both are worth their own issues; this PR takes
nativefrom "cannot build any GC-rooted function" to "builds and runs correctly", and stops the corpora from silently rotting again.Summary by CodeRabbit
New Features
Bug Fixes
Chores