semantic-diff

A "did the meaning change?" oracle for Rust CI. semantic-diff hashes each function's region-erased, pre-optimization monomorphized MIR to a 128-bit fingerprint that is stable across cosmetic edits and flips when the compiler would emit different code. The delta between two commits' fingerprint sets is a precise input to test selection, review attention, and re-audit scope.

We pay CI for the text diff. semantic-diff makes you pay for the meaning diff.

A commit that runs rustfmt, renames locals (even to different lengths), and adds doc comments moves zero fingerprints — nothing's meaning changed — while flipping one < to <= moves exactly the fingerprint of the one function that changed, and selects exactly the tests that reach it.

What it is — and what it is not

It is a regression-test-selection (RTS) oracle built on rustc's own stable hashing, at monomorphized-function granularity. Two clean builds of the same commit produce a byte-identical fingerprint set (under a stated precondition — a frozen toolchain + feature graph + remapped sysroot), so the diff between two commits is a portable, function-level "what could behave differently" set.

It is not "as correct as the compiler." The honest, defensible claim is "as discriminating as rustc's pre-optimization monomorphized MIR at a pinned toolchain + feature graph." Concretely, the honest boundary — measured, not hand-waved:

• The body hash misses some codegen-affecting changes (a const N: 7→8 a body references, a type-layout change, cfg/feature/panic/toolchain changes). These are caught by a separate build-config gate that forces "run everything," not by the per-function hash. When the tool can't represent something (e.g. a pure-library dependency it didn't monomorphize), it fails loud as INCOMPLETE → run the full suite rather than report a silent "nothing changed."
• Rewriting a for loop into an iterator chain is not free. We hash pre-optimization MIR, where a for loop and a .fold(...) are structurally different; they converge only after the optimizer, which we deliberately don't run. So that rewrite does move the fingerprint — a safe over-selection (a few extra tests run), never a missed change.
• Indirect calls (dyn, fn-pointers) widen selection, never silently skip. A test whose call graph hits an edge we can't follow statically always runs.

The guiding rule everywhere: the dangerous direction is a false UNCHANGED (a skipped test that should have run), so when in doubt the tool over-selects. See docs/prior-art.md for the full, adversarially-tested list of limitations.

Requirements (read this first)

This project builds and runs only inside a Linux container. The fingerprinting driver is a rustc codegen-backend plugin that links compiler internals (rustc_private: rustc_middle, rustc_data_structures, the StableHasher), which require a pinned nightly toolchain with the rustc-dev + rust-src components and RUSTC_BOOTSTRAP=1. The pinned nightly is load-bearing — StableHasher output and the rustc_private APIs drift across nightlies, so a fingerprint is only stable within a frozen toolchain.

• You need Docker. The Dockerfile pins the toolchain (nightly-2026-04-03); no --privileged or --cap-add is required.
• The pure index/diff/select engine (sdiff-core) and the sdiff CLI build on host stable Rust, but they only consume the .idx/.reach artifacts the container-only driver produces.
• There is deliberately no cargo install / Homebrew / curl | bash: a codegen-backend plugin isn't a host binary you drop on PATH. The "install" is the dev container.

Quickstart

# 1. Build the pinned-nightly dev container (one time).
docker build -t semantic-diff-dev .

# 2. Prove the central claim: the same commit built twice → byte-identical fingerprints.
docker run --rm -v "$PWD":/work -w /work semantic-diff-dev bash tests/m0_double_build.sh

Expected:

ADDED: 0  REMOVED: 0  CHANGED: 0
BYTE-IDENTICAL
M0 PASS: same commit built twice ⇒ byte-identical fingerprint set.

The full four-act walkthrough (cosmetic-robust, behavior-sensitive, honest about its blind spots, and the test-selection payoff) is in DEMO.md.

The proof (why this isn't just rustc incremental)

rustc's own query fingerprints are salted with StableCrateId (crate name + disambiguator + version + -Cmetadata) and are only ever compared within a single compiler session. They were never engineered to be equal across two independent clean builds. That cross-build equality — under a stated precondition — is the entire contribution, and it is proven live by a same-commit double-build yielding a byte-identical fingerprint set. Every correctness claim in the test suite ships a must-fail negative control (a deliberately-wrong variant the test must reject):

Run	Proves
tests/m0_double_build.sh	M0 gate: same commit, two build dirs, remapped paths → byte-identical fingerprints
tests/differential.sh	14 controls: cosmetic edits → CHANGED: 0; a <→<= edit → CHANGED: 1 naming only that function; layout / const / track_caller / async / ICE backstops all fire
tests/select.sh	test selection scopes to the changed function; a change behind &dyn widens (never a silent skip)
tests/workspace.sh	cross-crate: a generic dependency change is detected; a non-generic one fails loud as INCOMPLETE
tests/scale_regex_syntax.sh	the same pipeline on a real crate (see below)
cargo test	the pure sdiff-core engine + sdiff CLI, on host stable

All container gates run in CI (.github/workflows/ci.yml).

Measured at scale, honestly

We ran the whole pipeline on a real, third-party, generic-heavy crate with a real unit suite (regex-syntax 0.8.11, vendored + sha-pinned; see tests/scale_regex_syntax.sh) to answer what does this actually buy? The results are stated, not spun:

• It builds a real crate with no compiler crash (1448 monomorphized functions fingerprinted), and the same-commit double-build is byte-identical there too.
• Selection is sound at scale. For two real behavioral edits, the harness computes the ground-truth failing set (by running the real suite), then confirms sdiff select chose a superset — every truly-failing test is selected, including one edit whose failures span two modules.
• Precision is modest on a tightly-coupled crate. A single-function change selects ~54% of tests on average (~46% savings), with a hard 51% floor of tests that always run because their call graph crosses a dyn/fn-pointer edge.
• The economics are honest: on this fast unit suite, per-build fingerprinting overhead (~1.0 s) exceeds running the entire suite (~0.6 s). RTS pays off only when test execution time dominates compile time (slow/integration/IO suites) — not a fast pure-CPU unit suite.

How it works (in one paragraph)

A codegen-backend shim registers itself via -Zcodegen-backend, and in codegen_crate it clones each function's pre-optimization monomorphized MIR (before the mono collector steals it), region-erases + span/name-canonicalizes it, and stable-hashes it — folding in the instance's type-argument type_id_hash — into a 128-bit fingerprint, then delegates real codegen to LLVM so a binary is still produced. Per-#[test] reachability is a BFS over the monomorphized call graph. The mechanism (stable hashing of region-erased MIR, the codegen-backend seam, the call-graph worklist) is borrowed from rustc internals, rustowl, and cuda-oxide; the new work is cross-build stability engineering and the sound RTS argument. Full detail in docs/design.md.

The sdiff CLI

sdiff diff   <A> <B>                              # set-diff two fingerprint indexes (file or shard dir)
sdiff select <A> <B> <HEAD.reach> [--test-prefix P]  # print the minimal set of tests to run
sdiff --help | --version

Exit codes are a CI contract: diff exits 0 iff the delta is empty AND config matches AND the set is complete (1 otherwise); select exits 0 for RUN NONE (a safe skip) and 1 when there is work; exit 2 is a usage/load error. sdiff-core is rustc_private-free and builds on host stable.

Repository layout

crates/
  sdiff-core/   index format + delta engine + test selection (pure Rust, host-testable)
  sdiff/        the `sdiff` CLI
  sdiff-driver/ rustc codegen-backend shim (container-only; computes fingerprints from MIR)
docs/
  spec.md       the hardened source of truth (what / why / the honest boundary)
  design.md     how it works (components, data flow, borrowed-technique adaptation)
  prior-art.md  prior art, how we differ, and the full measured-limitations list
fixtures/       demo crates + fixture variants + the vendored scale-measurement crate
tests/          the container gates + host-side scripts (all wired into CI)
DEMO.md         the four-act walkthrough

Documentation & contributing

• docs/spec.md — the source of truth: what it is, the hardened claim, the boundary.
• docs/design.md — architecture and mechanism.
• docs/prior-art.md — prior art, how we differ, and every measured limitation.
• DEMO.md — the live four-act demo.
• CONTRIBUTING.md — the discipline this project runs on (every correctness test ships a must-fail negative control), the dev-container loop, and what's in/out of scope.

License

Dual-licensed under either of MIT or Apache-2.0, at your option.