Feat/batched fri per epoch#768
Draft
diegokingston wants to merge 16 commits into
Draft
Conversation
Round 1 now commits all tables' main-split LDE matrices into ONE mixed-height MixedMmcs and absorbs a single root (before the shared LogUp challenges), replacing the per-table main-root absorption. Single-table roots are byte- identical to the old per-table row-pair tree, so single-table proofs are unchanged; only multi-table transcripts change, so multi-table prove->verify tests are #[ignore]'d until the verifier is updated (Scope B Task 7). The MMCS is built transiently for the root (Task 5 will thread it into round 4 and drop the per-table trees).
Absorb ONE mixed-height MMCS root over every aux-carrying table's LDE into the shared transcript (after Phase B LogUp challenges, before forking), replacing the per-fork per-table aux root absorptions. Mirrors the main-trace batched commit. Transient MMCS (root only); Round 4 still opens per-table aux trees. Single-table roundtrips byte-identical (131 passed / 17 ignored).
Drop the per-table transcript forks. multi_prove now runs one linear transcript: bus contributions bind, then per-table beta (sequential), then round 2 composition built in parallel and committed as ONE mixed-height MMCS root, then round 3 (z, OOD) and round 4 (FRI) per table sequentially with PER-TABLE z. prove_rounds_2_to_4 is inlined and removed. Round 4 is still per-table (per-table FRI + per-table StarkProof/MultiProof) — the single batched FRI and shard proof format are the next step. Multi-table verify stays gated to the verifier task; single-table roundtrips remain byte-identical (131 passed / 17 ignored, clippy clean) because for one table the linear transcript equals the old fork and the single-matrix composition MMCS root equals the per-table composition root.
Sample a single z for the whole epoch instead of one per table, against the tallest table's domain. Since every shorter table's LDE and trace domain is a subgroup of the tallest's, z out-of-domain for the tallest is out-of-domain for all. Cleaner and simpler to mirror in the verifier; byte-identical for a single table (131 passed / 17 ignored, clippy clean).
Add the unified-shard proof types: BatchedMultiProof (shared main/aux/composition MMCS roots + one batched FRI + per-query MixedOpenings + per-table OOD data), BatchedQueryOpening (one shared auth path per phase per query), BatchedTableData. Make MixedOpening serde-serializable. Additive/pub — compiles clean, no behavior change yet; the prover (multi_prove_batched) and verifier wire these next.
Compute every table's round-3 OOD and absorb it before starting round 4, instead of interleaving round 3/4 per table. Creates the shared 'after rounds 1-3' boundary the batched path will reuse, and is required for the batched round 4 (one FRI over all tables needs all OOD bound first). Reference MultiProof path unchanged in behavior; single-table byte-identical (131/0/17).
Extract rounds 1-3 into prove_rounds_1_to_3 (returns round1s/round2s/round3s, shared z, and the three KEPT MMCS trees + per-table lde heights). multi_prove (reference MultiProof path) reuses it unchanged. New multi_prove_batched runs the batched round 4: gamma (one shared DEEP challenge) -> per-table DEEP codewords (bit-reversed) -> absorb height histogram -> alpha -> combine_by_height -> batched_commit_phase (fold-and-inject FRI) -> grinding -> query indices vs the tallest domain -> per-query open_batch on main/aux/composition MMCS (one shared path each) + per-preprocessed-table precomputed openings -> BatchedMultiProof. Reuses Scope-A fri::batched primitives. Reference path byte-identical (131/0/17, clippy clean). No verifier yet, so no batched roundtrip. Also: module-level allow(private_interfaces) — surfacing the rounds-1-3 bundle through a pub trait method trips the lint across the whole internal trait; the round types are never nameable across the crate boundary.
Mirror multi_prove_batched: one linear transcript, one shared OOD point z, and one fold-and-inject FRI over the height-combined per-table DEEP codewords, with all tables opened per query from the three shared mixed-height MMCS trees. - batched_multi_verify + batched_synthetic_table_proof (reuses step_2 and reconstruct_deep_composition_poly_evaluation per table via a lightweight synthetic StarkProof carrying only the OOD/public fields those read). - Transcript replay: precomputed roots -> batched main root -> LogUp challenges -> batched aux root -> bus contribs -> per-table betas -> batched composition root -> shared z (tallest domain) -> per-table OOD -> gamma -> derive_batched_fri_challenges (histogram, alpha, betas, iotas). - MMCS verify_batch per phase with AIR-intrinsic width binding: main-split width = trace_columns - num_aux - num_precomputed (trace_layout().0 is a logical figure for step-packed AIRs, not the physical column count). - Fold-and-inject query check inverts batched_commit_phase exactly: initial fold of the tallest (uncommitted) layer, then per layer inject the tables entering at that height (beta^2 * combined[h]) before verifying/folding. Converts all 17 Task-7-gated multi-table tests to the batched path (multi_prove_batched_ram + batched_multi_verify); 148 pass, 0 ignored.
11 tamper tests + 1 valid-anchor over a bus-balanced 3-table padding epoch, exercising every batched_multi_verify path: main/aux/composition MMCS auth, MMCS width binding, fold-and-inject terminal + layer-root + layer-sym checks, composition-OOD (step 2), query-count guard, grinding nonce, bus balance. 160 stark lib tests pass; new file clippy-clean.
…c path Point the public prove/verify at the batched STARK path so the existing server benchmarks measure it (prove/verify time + proof size), and to restore multi-table correctness — the linear-transcript refactor left the reference multi_prove -> multi_verify broken for multi-table (verifier still forked). - VmProof.proof: MultiProof -> BatchedMultiProof (serde-compatible; external consumers deserialize opaquely). - prove(): Prover::multi_prove -> multi_prove_batched. - verify_with_options(): batched_multi_verify + per_table.len() + batched COMMIT-bus-balance replay (replay_transcript_phase_a_batched: precomputed roots + single main_root + shared LogUp z/alpha). - test_utils: multi_prove_batched_ram. Minimal prove/verify test helpers + lt_bus completeness helper switched to batched (validated: the 4 lt_bus completeness tests, which FAIL on the reference path, PASS on batched). Scope: exploratory prototype for measurement. Continuation (continuation.rs) stays on the reference path — its cross-epoch L2G binding reads PER-TABLE main roots that the unified shard replaces with one MMCS root (a real design question, deferred). Reference multi_prove/multi_verify kept alive for it.
Switch two positive multi-table roundtrips to the batched path as end-to-end validation of the default MMCS prover/verifier: - lt_bus completeness (LT sender/receiver): 4 tests that FAIL on the reference path now pass on batched. - bitwise honest (true preprocessed table via with_preprocessed + hardcoded commitment): exercises precomputed-root check + precomputed openings + DEEP base = precomputed ++ main-split — the preprocessed path the real VM uses. Together with the stark-crate roundtrip/soundness/mixed-height/different-blowup coverage, every constituent path of the monolithic batched prove/verify is validated locally (full 14-table ELF run needs the server toolchain).
Collaborator
Author
|
/bench |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.