feat: add sequencer subcommand with embedded store#2102
Draft
sergerad wants to merge 16 commits into
Draft
Conversation
…bedded-store-phase-3a
sergerad
commented
May 21, 2026
| @@ -0,0 +1,154 @@ | |||
| use std::collections::HashMap; | |||
Collaborator
Author
There was a problem hiding this comment.
@Mirko-von-Leipzig having server/embedded.rs doesn't really make sense. But I think potentially this whole block-producer crate might move into the sequencer crate? Or at least we need to just return to this on phase 4 / cleanup to decide how to properly structure it when we remove the RPC path for block producer.
sergerad
commented
May 21, 2026
| /// The two services expose an identical interface; the only difference is which proto package | ||
| /// hosts the service declaration. The embedded sequencer serves `sequencer.NtxBuilderApi` so | ||
| /// the ntx-builder binary can point its `--store.url` at the sequencer without depending on the | ||
| /// standalone store binary. |
Collaborator
Author
There was a problem hiding this comment.
Note this, another thing for phase 4 / cleanup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relates to #1961. Implements Phase 3a.
Adds a
miden-node sequencersubcommand that runs the store, block-producer, and RPC components in a single process, eliminating the gRPC hops between them for store reads, block application, and transaction submission.Some of these changes (enums for remote vs local store/block-producer) will be refactored / removed in Phase 4 (cleanup).
TLDR is:
miden-node sequencercommandsequencer.NtxBuilderApifor ntx-builder to point toChanges
Store crate — exposes the types needed for in-process use:
State,StoreApi,ProvenTipWriter,ApplyBlockError, and aserve_replicafunction that runs the proof scheduler alongside a newreplica.ApigRPC service (see below).replica.proto— new internal proto that promotesstore.StoreReplicaout ofstore.protointo its ownreplica.Apiservice with its own descriptor. This decouples the replica streaming interface from the store's client-facing proto surface.sequencer.proto— new internal proto that promotesstore.NtxBuilderout ofstore.protointo its ownsequencer.NtxBuilderApiservice with its own descriptor.Block-producer crate — adds
LocalStoreClient(wrapsArc<State>directly) alongside the existing gRPC-based client, unified under aStoreClientenum. AddsEmbeddedBlockProducer, a parallel struct toBlockProducerthat takesArc<State>instead of a store URL.EmbeddedBlockProducer::start()returns aBlockProducerHandle— a cloneable in-process handle that bypasses gRPC for transaction and batch submission — together with a future that drives the gRPC server, batch builder, and block builder.RPC crate — adds
StoreBackendenum (Remote(StoreRpcClient)|Embedded(Arc<StoreApi>)) soRpcServicecan dispatch store reads in-process. AddsBlockProducerBackendenum (Remote(BlockProducerClient)|Embedded(BlockProducerHandle)) so transaction submission is routed directly to the in-process mempool without a loopback gRPC round-trip. AddsEmbeddedRpc, a parallel struct toRpcthat takesArc<StoreApi>andOption<BlockProducerBackend>instead of URLs.bin/node— addssequencer bootstrapandsequencer startsubcommands.startwires the three embedded components together:serve_replica,EmbeddedBlockProducer::start(), andEmbeddedRpcrun as concurrent tasks sharing a singleArc<State>. Three service endpoints are exposed: the client-facing RPC API, the block-producer API, and the replica streaming API.bin/ntx-builder— replaced--store.urlwith--sequencer.urlwith the intention being that only the sequencer services ntx-builder queries that are store related.What stays the same
The existing
store,block-producer, andrpcsubcommands are untouched — CI stays green throughout. The sequencer is purely additive.