Select cache values to verify by key fingerprint, not value fingerprint

[xmakro]

rustc verifies ~1/32 of values loaded from the incremental cache by re-hashing them. The subset is selected by the value fingerprint, so it changes whenever a value changes and between any two compiler builds. That makes verification failures flaky to reproduce, and it moves the re-hashing cost around between the two builds rustc-perf compares, which is why rustc-perf forces -Zincremental-verify-ich on every benchmarked invocation.

This PR selects by the key fingerprint and the session count instead. Both are fixed for a given previous session, so re-running a failed build verifies the same nodes and a verification failure reproduces on retry. to_smaller_hash mixes both fingerprint halves because neither half is evenly distributed on its own: DefPathHash keys share the StableCrateId, HirId keys carry a sequential id.

The per-session sampling rate is unchanged at 1/32, and -Zincremental-verify-ich is unchanged. Coverage over time improves: the subset rotates with the session count and sweeps the entire cache every 32 sessions, while under value selection a node whose value never changed kept its roll forever, so 31/32 of unchanging values were never verified on a given toolchain.

Between two compiler builds the subset still differs, because key fingerprints incorporate the rustc version through StableCrateId. Pinning the version via RUSTC_FORCE_RUSTC_VERSION and RUSTC_OVERRIDE_VERSION_STRING makes corresponding sessions identical; session counts align because both builds run the same scenario sequence from a fresh incremental directory. That would let rustc-perf stop forcing full verification and measure the incremental configuration users actually run.

Coverage across compiler releases is unchanged: the version salt reshuffles the subset each release under either scheme, and compiletest forces full verification in all incremental tests.

[xmakro]

r? @cjgillot
cc @nnethercote

[cjgillot]

That would let rustc-perf stop forcing full verification and measure the incremental configuration users actually run.

I'm not sure this is a goal. Having rustc-perf force fingerprinting gives an upper bound and may detect bottlenecks that we would otherwise miss.

@bors try @rust-timer queue

r=me if perf is green

[rust-bors]

☀️ Try build successful (CI)
Build commit: db3d63f (db3d63fe29bc48543b30f9ba77be28760a8f96c7)
Base parent: 0b63def (0b63defc8fd68957c635fe61f097b84df8fd9611)

[xmakro]

The motivation is a planned follow-up to #160214 that carries unchanged query cache values forward as raw bytes, without decoding them. That changes two things:

1. Recomputation no longer re-hashes those values, so sampled verification is the only check their bytes ever get. Value selection pins a value's roll forever, so 31/32 of them would never be checked. Session rotation covers everything within 32 sessions.

2. Forcing full verification makes rustc-perf decode and re-hash every carried value, which is exactly the work carry removes, so the benchmark would show near zero for a real win in the configuration users run. Deterministic selection removes the noise that forced the flag in the first place.

Keeping a forced-verification run as an upper bound seems fine. I just want a sampling scheme that a user-matching configuration could rely on.

[xmakro]

That would let rustc-perf stop forcing full verification and measure the incremental configuration users actually run.

I'm not sure this is a goal. Having rustc-perf force fingerprinting gives an upper bound and may detect bottlenecks that we would otherwise miss.

@bors try @rust-timer queue

r=me if perf is green

A motivation would be for a follow-up to #160214 where we carry unchanged query cache values forward as raw bytes, without decoding them. Forcing full verification makes rustc-perf decode and re-hash every carried value, which is the work carry removes, so the benchmark would show no win. In this case, it would be good if rustc-perf resembled what users are running.

[rust-timer]

Finished benchmarking commit (db3d63f): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking means the PR may be perf-sensitive. Consider adding rollup=never if this change is not fit for rolling up.

@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This perf run didn't have relevant results for this metric.

Max RSS (memory usage)

Results (primary -0.2%, secondary -0.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.4%]	1
Regressions ❌ (secondary)	0.4%	[0.4%, 0.4%]	1
Improvements ✅ (primary)	-0.9%	[-0.9%, -0.9%]	1
Improvements ✅ (secondary)	-0.7%	[-0.9%, -0.6%]	2
All ❌✅ (primary)	-0.2%	[-0.9%, 0.4%]	2

Cycles

Results (primary -0.1%, secondary 0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.6%	[0.4%, 0.9%]	6
Regressions ❌ (secondary)	0.6%	[0.4%, 1.1%]	6
Improvements ✅ (primary)	-0.8%	[-1.2%, -0.6%]	6
Improvements ✅ (secondary)	-0.7%	[-0.8%, -0.6%]	4
All ❌✅ (primary)	-0.1%	[-1.2%, 0.9%]	12

Binary size

Results (primary -0.1%, secondary -0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.1%	[-0.1%, -0.0%]	74
Improvements ✅ (secondary)	-0.1%	[-0.1%, -0.0%]	57
All ❌✅ (primary)	-0.1%	[-0.1%, -0.0%]	74

Bootstrap: 489.577s -> 488.977s (-0.12%)
Artifact size: 390.24 MiB -> 390.21 MiB (-0.01%)

[cjgillot]

@bors r+

[rust-bors]

📌 Commit 012c356 has been approved by cjgillot

It is now in the queue for this repository.

🌲 The tree is currently closed for pull requests below priority 10. This pull request will be tested once the tree is reopened.

Reason for tree closure: manually handling queue due to backlog