ci(deploy): reproducible build - #453
Draft
KSXGitHub wants to merge 8 commits into
Draft
Conversation
The release binary must be determined by the source code and the pinned toolchain alone, not by the host that performs the build. Two host-specific inputs previously leaked into the artifact. First, the dependency source paths from the cargo home were embedded verbatim into the binary through the `file!()` macro, so the absolute registry path of the build machine became part of the output. A new `ci/github-actions/reproducible-build-env.sh` script, run before each build in the three deployment build jobs, sets `--remap-path-prefix` for the cargo home and the workspace. This replaces those prefixes with fixed placeholders and is the stable equivalent of the still-unstable `-Z trim-paths` option. The script also pins `CARGO_INCREMENTAL=0` and provides `SOURCE_DATE_EPOCH`. Second, the binary was stripped by a separate `strip` step whose output depends on the version of the host `strip` program. The release profile now sets `strip = "symbols"`, so the toolchain strips the binary while linking, and the redundant strip steps are removed. This also strips the Windows binaries, which the removed steps never covered. Parallel code generation is kept. On a fixed toolchain it is already deterministic, verified by two independent builds producing byte-identical binaries, so `codegen-units` stays at its default rather than being serialized to 1. The value is pinned to preserve that parallelism against a future change to the default. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013UjrvAEQT3ryDqfspssN3g
Explain that release binaries are reproducible, that parallel code generation is kept because it is deterministic on a fixed toolchain, and how the `strip = "symbols"` profile setting and the `--remap-path-prefix` script achieve this. Include the commands to reproduce a binary locally and compare its checksum. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013UjrvAEQT3ryDqfspssN3g
Performance Regression Reportscommit: a92e74b There are no regressions. |
KSXGitHub
commented
Jul 25, 2026
KSXGitHub
left a comment
KSXGitHub
left a comment
Owner
Author
There was a problem hiding this comment.
Claude Code,
Below are many different reviews. Each may belong to different categories.
I wish that you do not collapse all the unrelated tasks into a single commit.
You shall analyze the reviews. You shall make groups of related reviews.
Each group of related reviews correspond to a task and a git commit.
A git commit should have its purpose clear and focused.
If there would be multiple groups of related reviews, then I expect multiple commits.
The pin was not required for reproducibility. Because the toolchain is pinned, the release build uses the same default codegen-units on every host, and that default is already deterministic. Verified by building the same sources with the pin removed at `-j1`, `-j16`, and `-j64`, and under a restricted CPU affinity, each producing an identical binary. The default also keeps parallel code generation, so removing the pin does not serialize the build. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013UjrvAEQT3ryDqfspssN3g
Remove the paragraph and the bullet list that restated what the `Cargo.toml` profile and the reproducible-build script already document in their own comments. The section now states the reproducibility guarantee and the way to verify it, without duplicating the mechanism details. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013UjrvAEQT3ryDqfspssN3g
Move the local reproduction commands out of `CONTRIBUTING.md` and into `verify-reproducible-build.sh`, alongside `run.sh`, `test.sh`, and `generate-completions.sh`. The script rebuilds a release binary with the reproducible environment, mirroring the deployment workflow including the explicit target, and prints its checksum for comparison against a published release. The target defaults to the primary Linux target and can be overridden. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013UjrvAEQT3ryDqfspssN3g
The standard library embeds its own source paths into the binary through panic locations. They normally read `/rustc/<commit-hash>/...`, but when the `rust-src` component is installed the compiler rewrites them to the local toolchain directory under the sysroot, which differs from one machine to the next. The reproducible-build environment remapped the cargo home and the workspace but not the sysroot, so a build on a machine with `rust-src` installed produced a different binary from one without it. Add a third `--remap-path-prefix` that maps the sysroot rust source directory back to the canonical `/rustc/<commit-hash>` form, in both the deployment environment script and `verify-reproducible-build.sh`. Verified by building with `rust-src` installed: the result is now byte-identical to the build without it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013UjrvAEQT3ryDqfspssN3g
The GNU target links its C runtime, the glibc startup objects and the GCC version string, from the build environment rather than from the Rust toolchain. A bare host build therefore varies between distributions even after paths are remapped, so two hosts produce different binaries. Add `reproducible-build.sh`, which builds a release binary inside a container image pinned by digest. The image ships rustc 1.97.1 on Debian bullseye, so the GCC and glibc are fixed and the result is identical on every host. Bullseye's glibc 2.31 keeps the required glibc version low for broad compatibility, and the binary remains an ordinary dynamically linked ELF rather than being tied to any special filesystem. The script detects Docker or Podman, forwards proxy variables for networks that need them, and mounts the workspace read-only. The `build_linux` job now calls this script instead of installing a toolchain and building on the runner, and `verify-reproducible-build.sh` delegates to it so a local checksum matches the released artifact. The macOS and Windows jobs are unchanged, since a Linux container cannot build them. Verified in a container for both targets: `x86_64-unknown-linux-gnu` and `x86_64-unknown-linux-musl` each build byte-identically across repeated runs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013UjrvAEQT3ryDqfspssN3g
Update the reproducible-builds section to describe the pinned container image as the source of host independence, and note that the verification script needs Docker or Podman and accepts a target argument. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013UjrvAEQT3ryDqfspssN3g
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.