Skip to content

refactor(dash-spv): network manager refactor and sync pipelines optimized #902

Draft
ZocoLini wants to merge 2 commits into
devfrom
refactor/network-mod
Draft

refactor(dash-spv): network manager refactor and sync pipelines optimized #902
ZocoLini wants to merge 2 commits into
devfrom
refactor/network-mod

Conversation

@ZocoLini

@ZocoLini ZocoLini commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Incredible big PR that does many things to get to a point where the sync time is as low as possible, I will be working on refactoring and cleaning the introduced code in the next PRs but since we need a fast sync for a good user experience at the iOS release I would like to get this merge as long as it can sync

  • New dash-spv-bench crate that I used tot set different scenarios and be able to optimice and validate my changes|
  • New non-pub NetworkManager that is able to efficiently distribute work
  • Mechanism to connect to extra peers when under heavy load
  • Sync pipelines no longer wait for item to be stored to disc, they step forward as soon as the item they need is in memory
  • NetworkMessages enums fields in dashcore are being stored Behind a Box structure, to reduce memory consumption during sync
  • Items are stored to disc as soon as they arrive to avoid high memory usage

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 44e4cddd-74cd-405f-b87d-d3edd97ac9ed

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/network-mod

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.11765% with 78 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.80%. Comparing base (19690d3) to head (726da21).
⚠️ Report is 1 commits behind head on dev.

Files with missing lines Patch % Lines
dash-spv/src/network/peer.rs 84.67% 42 Missing ⚠️
dash/src/network/message.rs 83.33% 12 Missing ⚠️
dash-spv/src/network/discovery.rs 64.00% 9 Missing ⚠️
dash-spv/src/main.rs 0.00% 4 Missing ⚠️
dash-spv/src/sync/block_headers/manager.rs 97.72% 2 Missing ⚠️
dash-spv-ffi/src/client.rs 75.00% 1 Missing ⚠️
dash-spv/src/client/lifecycle.rs 92.85% 1 Missing ⚠️
dash-spv/src/client/queries.rs 0.00% 1 Missing ⚠️
dash-spv/src/network/mod.rs 95.45% 1 Missing ⚠️
dash-spv/src/sync/blocks/pipeline.rs 96.55% 1 Missing ⚠️
... and 4 more
Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #902      +/-   ##
==========================================
- Coverage   74.54%   73.80%   -0.75%     
==========================================
  Files         327      318       -9     
  Lines       75032    72110    -2922     
==========================================
- Hits        55936    53222    -2714     
+ Misses      19096    18888     -208     
Flag Coverage Δ
core 77.31% <83.33%> (+0.01%) ⬆️
ffi 49.68% <85.71%> (-1.27%) ⬇️
rpc 20.00% <ø> (ø)
spv 91.80% <94.78%> (+0.59%) ⬆️
wallet 74.41% <ø> (ø)
Files with missing lines Coverage Δ
dash-spv-ffi/src/callbacks.rs 83.21% <100.00%> (-0.37%) ⬇️
dash-spv/src/client/core.rs 63.04% <100.00%> (+0.82%) ⬆️
dash-spv/src/client/event_handler.rs 93.76% <100.00%> (-0.13%) ⬇️
dash-spv/src/client/events.rs 100.00% <100.00%> (ø)
dash-spv/src/client/mod.rs 100.00% <100.00%> (ø)
dash-spv/src/client/transactions.rs 100.00% <100.00%> (+10.00%) ⬆️
dash-spv/src/lib.rs 46.66% <ø> (ø)
dash-spv/src/network/manager.rs 78.02% <ø> (+5.23%) ⬆️
dash-spv/src/storage/mod.rs 86.15% <ø> (ø)
dash-spv/src/sync/block_headers/pipeline.rs 93.50% <100.00%> (-0.72%) ⬇️
... and 31 more

... and 18 files with indirect coverage changes

A docker-driven benchmark that syncs the SPV client against a frozen chain
snapshot (local peers via tc netem) or the real testnet/mainnet, reporting
sync time and a live dashboard. Base branch for the network-rewrite PR stacked
on top; it drives the rewritten client and does not build on dev alone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CUb3bkX9C1gBFA65GFsN53
@ZocoLini
ZocoLini force-pushed the refactor/network-mod branch from 39b7db2 to fd614fc Compare July 21, 2026 15:07
Replaces the old network layer with a self-coordinating PeerNetworkManager
(broker): it owns request de-duplication, pacing, per-peer in-flight sizing,
timeouts, retries and peer hot-swap, so the sync pipelines just declare what
they want. Highlights:

- Restore sync/ from dev and re-wire it to the broker; delete the per-pipeline
  DownloadCoordinator.
- Minimal `NetworkManager` trait + in-memory `MockNetworkManager`; the client is
  generic over the network (`DashSpvClient<W, N, S>`) with the network injected.
- Strict-priority message scheduler: control, then blocks, filters, filter
  headers, block headers.
- Drop peer storage and the peer-reputation system (unused; to be redone).
- Un-Box the `NetworkMessage` variants to stay close to dev.
- Storage kept as on dev (minus the removed peer storage).
- Restore dev's integration tests and the network-dependent unit tests via the mock.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CUb3bkX9C1gBFA65GFsN53
@github-actions

Copy link
Copy Markdown
Contributor

This PR has merge conflicts with the base branch. Please rebase or merge the base branch into your branch to resolve them.

@github-actions github-actions Bot added the merge-conflict The PR conflicts with the target branch. label Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-conflict The PR conflicts with the target branch.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant