Skip to content

backport: assumeutxo M2 — EvoDB multi-chainstate isolation and subsystem gating - #7456

Draft
PastaPastaPasta wants to merge 13 commits into
dashpay:developfrom
PastaPastaPasta:assumeutxo/m2-evodb-roles
Draft

backport: assumeutxo M2 — EvoDB multi-chainstate isolation and subsystem gating#7456
PastaPastaPasta wants to merge 13 commits into
dashpay:developfrom
PastaPastaPasta:assumeutxo/m2-evodb-roles

Conversation

@PastaPastaPasta

@PastaPastaPasta PastaPastaPasta commented Jul 13, 2026

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

M1 (#7451) added AssumeUTXO snapshot persistence, but Dash stores deterministic masternode, quorum, MNHF, and credit-pool state in a shared EvoDB. Running the snapshot and background chainstates concurrently therefore requires independent EvoDB transaction state and markers, chain-aware Dash validation, and protection against emitting or signing from the wrong chainstate.

This is milestone 2 of the AssumeUTXO series. It supplies the Dash-specific multi-chainstate foundation required by the later background-completion and loadtxoutset milestones.

What was done?

  • Added stable NORMAL and SNAPSHOT EvoDB identities, each with an independent transaction overlay, root batch, and best-block marker while retaining one physical EvoDB.
  • Added WriteDerived for immutable block-derived records. Independently derived values must serialize identically, including values pending in the other chainstate's overlay.
  • Wired block connect, disconnect, replay, verification, flushing, and snapshot activation to the calling chainstate's EvoDB identity. Snapshot activation now refuses a missing EvoDB marker instead of attaching snapshot coins to inconsistent Dash state.
  • Protected shared mined-quorum commitments from cross-chainstate erasure.
  • Passed the validating chainstate through Dash special-transaction and quorum processing. MNHF, asset-unlock, and quorum lookups now evaluate membership relative to the caller's chain instead of implicitly borrowing the active chain.
  • Suppressed active-tip, wallet, UI, and deterministic-masternode notifications from background validation. BlockChecked remains ungated because it reports validation results rather than active-tip changes.
  • Avoided penalizing peers when this snapshot-backed or pruned node cannot serve otherwise plausible masternode-list or quorum-rotation history.
  • Disabled DKG participation and quorum signing while an active snapshot remains unvalidated, including a final guard at the signature-share production boundary and clear RPC/status reporting.
  • Canonicalized serialization of block-derived masternode-list diffs and MNHF signals so logically identical values cannot produce false EvoDB mismatches.

Review follow-ups (appended commits):

  • A WriteDerived mismatch is local EvoDB corruption, never evidence about the block. It now aborts the node with M_ERROR (matching the existing EvoDbInconsistencyMessage convention) instead of marking the block BLOCK_CONSENSUS-invalid and penalizing the relaying peer. A typed EvoDbInconsistencyError preserves that classification through the catch blocks on the miner, RPC, and MNHF-recomputation paths.
  • EvoDB access outside a BeginTransaction scope previously always bound to the NORMAL identity, so transaction-less consumers (RPC, mempool, miner, P2P serving) could not see snapshot-chain records pending in the SNAPSHOT overlay. CEvoDB now tracks a default identity that snapshot activation sets to SNAPSHOT and ResetChainstates resets; the background-completion milestone must reset it to NORMAL at marker promotion (TODO noted in code).
  • GetListForBlockInternal no longer fabricates an empty "initial snapshot" masternode list when a diff for a DIP3-active block is missing; it throws instead. The message deliberately carries the IsBlockDataUnavailableError sentinel and is deliberately a plain runtime_error rather than EvoDbInconsistencyError: at that layer a missing diff can be benign (pending in the other chainstate's unflushed overlay, e.g. while serving historical mnlistdiff), so it is reported as unavailable history without penalizing the requesting peer, and only definite mismatches abort the node. Scheduler-thread consumers (CActiveMasternodeManager::UpdatedBlockTip, governance trigger creation) catch it and skip the update, since an uncaught exception there would terminate the node.

The batch contains the original eight focused commits, a rebase fixture adaptation after the txindex removal, and four review follow-up commits. The partial Bitcoin Core BlockInfo and ChainstateRole prerequisites were intentionally moved to the later loadtxoutset milestone where their APIs are first consumed.

How Has This Been Tested?

  • Added focused EvoDB unit coverage for overlay isolation, tombstones, cross-overlay derived-value checks, independent marker flushing, disk mismatch rejection, and preservation of the legacy NORMAL marker key.
  • Added dual-chainstate integration coverage for restart consistency, snapshot-only flushing, shared quorum erasure, chain-aware commitment/quorum lookup, cache-order isolation, and suppression of background notifications.
  • Added canonical serialization coverage for deterministic masternode-list diffs.
  • Added transaction_less_access_uses_default_identity covering default-identity resolution of transaction-less reads and writes. Known gap: the DIP3-active missing-diff throw has no end-to-end unit test because regtest activates DIP3 at height 432, above the unit-test chain heights; it is covered by the serving-path catch and review.
  • Before the stack-only relocation of the two unused Bitcoin prerequisite commits, make check passed and the following functional subset passed: feature_mnehf.py, feature_asset_locks.py, feature_dip3_deterministicmns.py (both modes), feature_llmq_signing.py (both modes), feature_llmq_rotation.py, and rpc_quorum.py.
  • After the review follow-up commits: full rebuild plus evo_db_tests, evo_deterministicmns_tests, evo_mnhf_tests, evo_assetlocks_tests, and validation_chainstatemanager_tests pass (20 cases), and lint-circular-dependencies is clean.
  • Every rewritten commit passes git diff-tree --check.

Breaking Changes

No released-interface changes. Internally, ActivateExistingSnapshot becomes fallible so startup can reject a snapshot chainstate whose EvoDB marker is missing. Two RPC-visible additions while a snapshot is active and unvalidated: masternode status gains a quorumParticipation field and appends a "DKG participation and quorum signing disabled" clause to status, and quorum sign fails with an explanatory error. AbstractEHFManager::Signals changed from std::unordered_map to std::map (canonical serialization); the on-disk encoding is unchanged, but iteration order of EHF signals in getblockchaininfo output is now sorted.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation (the AssumeUTXO documentation update is part of a later milestone)
  • I have assigned this pull request to a milestone

@PastaPastaPasta

Copy link
Copy Markdown
Member Author

See https://gist.github.com/PastaPastaPasta/c5108775f7b126f88856cde20d7a8556 for ai generated commit by commit explanation.

@PastaPastaPasta
PastaPastaPasta force-pushed the assumeutxo/m2-evodb-roles branch 2 times, most recently from 06b5c6f to 1a95697 Compare July 14, 2026 17:01
@github-actions

Copy link
Copy Markdown

This pull request has conflicts, please rebase.

@github-actions

Copy link
Copy Markdown

This pull request has conflicts, please rebase.

Pass the validating chainstate through special transaction and quorum commitment processing instead of borrowing the active chainstate.

Interpret mined-commitment records and quorum resolution relative to the caller's chain. The cached values remain reusable, but chain membership is reevaluated across reorgs and chainstates while public non-validation callers retain active-chain semantics. This prevents snapshot-seeded records from suppressing commitments or satisfying MNHF and asset-unlock quorum lookups during background validation.

Add dual-chainstate coverage for a commitment seeded at a block not yet contained by the background chain, including HasQuorum and GetQuorum cache-order checks.
Emit block, tip, deterministic masternode-list, UI, and flush notifications only for the active chainstate. In particular, suppressing background ChainStateFlushed prevents a background locator from regressing wallet best-block state.

Keep BlockChecked ungated because its subscribers are mining/block-submit and peer validation/relay accounting; it does not reach CMNAuth. Document all 21 B3 call-site dispositions and extend the dual-chainstate test with validation-interface and UI counters.
Check local block-data availability before building masternode-list diffs and quorum rotation info. Treat failures caused by pruning or an unvalidated snapshot base like pruned getdata: log and silently drop the plausible request without increasing the peer's misbehavior score.

Malformed and implausible requests retain the pre-existing penalties.
Disable DKG participation and quorum signing until snapshot background validation completes. Enforce the refusal at CreateSigShare, the actual share-production boundary, so direct RPC, async, and queued signing paths cannot bypass it.

The quorum sign RPC now returns a clear JSON-RPC error for both submit modes, and masternode status exposes the disabled participation state. Add unit coverage for the shared production-gate predicate across snapshot activation.
@PastaPastaPasta
PastaPastaPasta force-pushed the assumeutxo/m2-evodb-roles branch from 26c8718 to b6edcf8 Compare August 1, 2026 18:17
PastaPastaPasta and others added 4 commits August 1, 2026 16:10
A WriteDerived failure means independently derived block data disagrees with the copy already recorded in EvoDB. That is local state corruption (or a cross-chainstate divergence bug), never evidence about the block being processed. Previously the mismatch surfaced as BLOCK_CONSENSUS: the block was persistently marked BLOCK_FAILED_VALID (surviving restart and forking the node off the network) and the relaying peer was handed a 100-point misbehavior score via BlockChecked, which background validation also triggers.

Instead, follow the existing EvoDbInconsistencyMessage convention: request node shutdown via AbortNode and fail validation with M_ERROR, which neither marks the block invalid nor punishes peers. The credit-pool and MNHF sites abort at the throw site because miner and RPC callers never pass through a validation-state catch; a typed EvoDbInconsistencyError lets the four block-path catch blocks that would otherwise swallow it into BLOCK_CONSENSUS reclassify it as M_ERROR.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ethrow

CMNHFManager::GetForBlock re-throws internal ProcessBlock failures as a plain runtime_error, which would let a downstream generic catch misreport an EvoDB mismatch (M_ERROR) as a consensus failure. Re-throw typed when the validation state carries M_ERROR.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reads and writes outside a BeginTransaction scope previously always bound to the NORMAL identity. While a snapshot chainstate is active, transaction-less consumers (RPC, mempool, miner, P2P serving) could not see snapshot-chain records still pending in the SNAPSHOT root overlay, and transaction-less writes dirtied the wrong overlay. CEvoDB now tracks a default identity which snapshot activation (ActivateSnapshot, ActivateExistingSnapshot) sets to SNAPSHOT and ResetChainstates resets; snapshot completion must reset it to NORMAL when marker promotion lands.

Also make the GetListForBlockInternal fallback loud: a missing list diff for a DIP3-active block is pending-elsewhere or corrupt data, never the pre-DIP3 genesis of the masternode list, so throw instead of silently caching an empty list and clobbering m_initial_snapshot_index. The thrown message carries the IsBlockDataUnavailableError sentinel and BuildSimplifiedMNListDiff converts it into a serve failure, so peers requesting such history are not penalized.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GetListForBlock can now throw when list data is unavailable (missing diff for a DIP3-active block). CActiveMasternodeManager::UpdatedBlockTip and GovernanceSigner's trigger creation run on the scheduler thread, where an uncaught exception terminates the node; catch it there and skip the update instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@PastaPastaPasta
PastaPastaPasta force-pushed the assumeutxo/m2-evodb-roles branch from 11a31da to 5f3a876 Compare August 1, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant