perf(drive-abci): stop rewriting the whole platform state every block

[PastaPastaPasta]

Issue being fixed or feature implemented

The saved platform state is written to GroveDB aux storage on every block, and on mainnet it is 1.28 MB — almost all of it masternode lists, validator sets and the chain-lock and instant-lock quorum sets.

PlatformSerializable::serialize_to_bytes also did self.clone() before converting to the saving form, so a block paid two full deep copies of some 4,000 masternodes plus the validator sets, then serialized 1.28 MB, then wrote it.

Measured replaying mainnet, per block: 1.08 ms serializing, 0.23 ms for the extra clone, and 1.28 MB into the block's transaction — 545 GB of writes over a full sync.

Stacked on #4570, which introduces the utils::is_historical_block predicate this uses.

What was done?

Three things.

Serialize from a borrowed state. A TryFrom<&PlatformState> for the saving form clones each field once instead of cloning the whole state first. A test asserts the bytes are byte-identical to the owned path.

Rewrite the full record only when it changed. The heavy fields carry a dirty flag, set by the accessors that can change them. While replaying history the full record is written only when the flag is set; a small companion record under a second aux key carries the per-block fields — block info, quorum hashes, protocol versions — every block. Both are written in the block's transaction, so a reader never sees them disagree, and a database without the companion record reads exactly as before.

Once the node is at the tip the full record is written every block again, so an up-to-date node always has a complete record on disk and an older drive-abci can still read it. Skipping is confined to a node that is catching up, where the remedy for any format trouble is the resync it is already doing.

Don't take a mutable borrow when nothing moved. Core reports the same quorums on most blocks and an empty masternode diff often, but update_quorum_info and update_state_masternode_list reached for _mut() accessors regardless — and the borrow alone marks the state dirty. Both now decide read-only whether anything actually changed first.

How Has This Been Tested?

Full mainnet replay, genesis to 424,981. Every committed app hash matched a reference sync across all 424,971 heights, and the final app hash matched exactly.

Crash recovery specifically: two mid-sync restarts, at heights 40,200 and 80,188. Both resumed at the right height from the companion record and continued with no app-hash mismatch — which requires the heavy fields to be correct too, since they feed the app hash through masternode identity updates and rewards.

cargo test -p drive-abci --lib — 2,776 passed, including the new byte-identity test.

Breaking Changes

None for normal operation, an interrupted sync included. A node that stops mid-sync restarts and carries on by itself with no operator action: the companion record restores the height, and the heavy fields are unchanged since the full record was written. Two mid-sync restarts are part of the testing above.

The one case that needs care is a deliberate downgrade. A node interrupted mid-initial-sync leaves a full record that lags the database, and an older drive-abci reading that database panics in the Info handler, which compares the saved state's app hash against GroveDB's root hash — confirmed by running a pre-change binary against a database this one wrote. That is why the skip is confined to historical blocks: a node that has caught up has a complete record on disk and can always be rolled back. A node caught mid-sync would need to finish syncing on the new build, or resync on the old one.

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated relevant unit/integration/functional/e2e tests
• I have made corresponding changes to the documentation

For repository code-owners and collaborators only

• I have assigned this pull request to a milestone

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 65f9a2c0-62a6-494b-95c1-16b35732d961

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[thepastaclaw]

🔍 Review in progress — actively reviewing now (commit a319483)
Stage: Phase-1 GLM Flash review → Sol gate verification
ETA: complete ~01:55 UTC (median 1h 11m across 30 recent reviews)
Running 4h 48m · Last checked: 2026-09-07 01:50 UTC

[PastaPastaPasta]

Review

Verdict: merge after changes, and after #4570 lands (this is stacked on it and its base will move).

1. Correctness

I audited the dirty flag. Every mutation of the six heavy fields goes through a PlatformStateV0Methods setter or _mut accessor; there is no direct field access outside the module, and the only PlatformState { .. } constructors start dirty. set_current_protocol_version_in_consensus marking dirty is the important subtle case, because the protocol version selects the on-disk structure. All readers go through fetch_platform_state, which applies the small record only when its height is at least the full record's height; both are written in the same transaction, so they cannot disagree. The Option<u64> comparison handles the pre-first-block case (None >= None) correctly.

The two "do not borrow mutably when nothing moved" changes are equivalent to the old code. The read-only quorum comparison checks equal length plus every current quorum present with the same index, which means the sets are identical. The empty-diff early return in update_state_masternode_list skips only no-op operations.

Local verification on a319483d: cargo test -p drive-abci --lib passes, and the four stop_and_restart strategy tests pass (they run with store_platform_state: true and the 2023 genesis time, so they do exercise the skip-and-merge path).

The downgrade hazard is real and well described. Confining the skip to historical blocks is the right mitigation.

2. Clarity

Clear. The three-part "What was done" maps onto the diff. The comment blocks in store_platform_state and recent.rs explain the invariant a future reader needs: both records in one transaction, full record rewritten every block at the tip.

3. Codebase standards

Two deviations:

• The two new Drive methods are not versioned. Every other Drive method, including store_platform_state_bytes right next to them, has a mod.rs dispatcher over a FeatureVersion in rs-platform-version and a v0/ implementation. store_platform_state_recent_bytes and fetch_platform_state_recent_bytes are plain impl Drive functions in platform_state/mod.rs. Please follow the convention.
• The bincode config for the small record is duplicated in store_platform_state and fetch_platform_state. Give PlatformStateRecent serialize_to_bytes / deserialize methods and keep the config in one place.

Also: TryFrom<&PlatformState> duplicates TryFrom<PlatformState> field by field. If the owned conversion has no production caller left, delegate it to the borrowed one and drop the duplicate.

4. Importance and alternatives

1.3 ms of a ~7 ms block and 545 GB of write amplification over a sync. Splitting the record into a stable full part and a small per-block part, without changing the full record's format, is the least disruptive design; splitting the on-disk format itself would break older readers. No simpler approach avoids the per-block write, since block info changes every block.

5. Missing test

There is no focused test of the merge path: store with the dirty flag set, mutate block info, store again with the flag clear on a historical block, fetch, and assert the block info is new and the heavy fields intact. The restart strategy tests cover it indirectly but only run on push and nightly.

I will push: rebase onto the fixed #4570, versioned Drive methods, PlatformStateRecent (de)serialization helpers, the dedupe, and a merge test.

---

🤖 Posted autonomously by Claude on behalf of pasta.

[PastaPastaPasta]

Rebased onto the new head of #4570 (9dfffa6) and pushed one commit on top of the original:

• Versioned Drive methods. store_platform_state_recent_bytes and fetch_platform_state_recent_bytes now follow the crate's mod.rs dispatcher + v0/ layout like store_platform_state_bytes next to them, with fetch_platform_state_recent_bytes / store_platform_state_recent_bytes feature versions in DrivePlatformStateMethodVersions (0 in drive versions v1 to v9 and the v2 test mock).
• One place for the small record's encoding. PlatformStateRecent::serialize_to_bytes / deserialize own the bincode config and error mapping; store_platform_state and fetch_platform_state call them instead of repeating both.
• No duplicated conversion. The owned TryFrom<PlatformState> for the saving struct delegates to the borrowed one. Only tests use the owned path now.
• A test for the merge. v0_historical_block_with_clean_heavy_fields_reloads_from_the_small_record stores a historical block that adds a masternode (dirty, written in full), then one that changes nothing heavy (clean, small record only), reloads through fetch_platform_state, and checks the block info comes from the second block and the masternode from the first. It also reads the raw full record and checks it is still the first block's, which is what proves the second block skipped the rewrite rather than repeating it.

cargo test -p drive-abci --lib -- platform_state update_state_cache update_quorum_info update_state_masternode_list: 15 passed (the full --lib suite passed on the pre-rebase head, 2,777). stop_and_restart strategy tests: 4 passed. fmt and clippy clean across drive-abci, drive and platform-version.

One thing to be aware of: the Tests workflow only triggers for PRs whose base is master, v*-dev or ci/*, so while this PR targets perf/checkpoint-skip-during-replay no Rust test has run in CI for it at any point (only the linter and Kotlin jobs show up). The numbers above are local. Once #4570 merges, retargeting this PR to v4.2-dev will run the full suite before merge.

Ready for human review once #4570 is in; this PR should be merged after it.

---

🤖 Posted autonomously by Claude on behalf of pasta.