Skip to content

Browser-native Bitcoin node: architecture and plan #1

Description

@melvincarvalho

Summary

Grow bitcoin-kernel from a consensus test runner into a browser-native Bitcoin node: validate Bitcoin's consensus rules in a tab, persist the chain to OPFS, and sync through a thin local bridge. Usable as a wallet in minutes; fully validated in the background within hours.

The hard part, the consensus engine, already exists in this repo: it validates headers, difficulty, blocks, scripts and SPV proofs against real test vectors, with real ECDSA/Schnorr. This issue captures the architecture, an honest performance model, and the milestones to get from here to "node".

Product idea: usable first, trustless later

Start at the chain tip and work backwards so the user has a working wallet almost immediately, then harden to full validation in the background. This stacks two proven designs:

  • Neutrino (BIP 157/158) for the instant wallet: headers-first to tip, then walk backwards pulling compact block filters and only downloading the few blocks that match your addresses.
  • assumeutxo for the background hardening: load a hash-checked UTXO snapshot, be usable in minutes, and validate genesis to snapshot in the background.

Trust ladder (the UI MUST be honest about which rung the user is on):

  1. Headers only, trust accumulated proof of work (most-work chain)
    • compact filters / SPV, light-wallet trust: find and verify your own coins
    • assumeutxo snapshot, trust one hash temporarily, usable now
    • background backfill complete, fully validated and trustless

Note: the wallet scan goes backwards (cheap via filters), but consensus validation goes forwards from a snapshot, because the UTXO set only exists by applying blocks in order. Two motions, stacked.

Architecture: three pillars

  1. Storage, OPFS. The enabler is createSyncAccessHandle() in a Web Worker (synchronous file I/O). Holds the ~650 GB of flat block/undo files, headers, filter data, and periodic UTXO snapshots. Requires navigator.storage.persist() so the browser does not evict it. IndexedDB is a trap (async, per-op overhead) and must not be in the hot path.
  2. Compute, WASM. WASM-SIMD libsecp256k1 for signature verification, one verifier per core via Web Workers, ideally zero-copy over SharedArrayBuffer (needs COOP/COEP headers, which we control). Multi-buffer SIMD SHA256 for hashing. The UTXO set lives in RAM (a custom compact hash map), not OPFS, with OPFS used only for durable snapshots.
  3. Networking, a local shim. A browser tab cannot open raw TCP to port 8333. A small local WebSocket-to-TCP bridge relays bytes to real peers; the tab still validates everything itself, so it stays trustless. The engine already speaks p2p over WebSocket (ws.js/p2p.js).

What already exists (engine)

  • Consensus validation: headers, difficulty (incl. testnet4 timewarp), blocks, transactions, scripts (Bitcoin Core's vectors), SPV merkle proofs.
  • Real verifyEcdsa / verifySchnorr (not stubs).
  • p2p.js, node.js, ws.js (p2p over WebSocket), filters.js (BIP158 compact filters), storage, mine.js, wallet.js.
  • The whole thing is pure ESM, zero-dependency, importable as bitcoin-kernel, and already runs validation client-side (the demo).

What is missing / to build

  • WASM-SIMD secp256k1: the engine's crypto is pure JS today (fine for the 1,191 test vectors, ~20-100x too slow for IBD). This swap is the prerequisite for everything below.
  • OPFS storage backend: flat block/undo store + UTXO snapshot format + crash recovery.
  • In-RAM UTXO set: compact hash map in WASM/SharedArrayBuffer, plus a signature cache.
  • Local TCP shim: ~100-line WebSocket-to-TCP bridge, runnable on the same machine.
  • assumeutxo: snapshot load, then background backfill genesis to snapshot.
  • Wallet path: headers-first sync + backwards filter scan + your-coins UTXO/history.
  • Long-run host: installed PWA / persistent worker so a multi-hour sync is not a casual tab.

Performance model (modeled, treat as +/- 2x until benchmarked)

All numbers assume the WASM-secp swap is done and the chain data is sourced fast. Hardware target: a high-end desktop (16-32+ cores, 64 GB RAM, 4 TB NVMe).

Mode Time to usable Trust
Light wallet (Neutrino) minutes PoW headers + filters
assumeutxo snapshot minutes, backfill behind one hash, temporarily
assumevalid (skip ancient sigs) ~2-3 h a recent checkpoint
Full verify-everything, ~1.3x native (future WASM wide-arith) ~3-6 h nothing
Full verify-everything, ~2x native (today's WASM-SIMD) ~5-10 h nothing

Why: ~2.5 billion signature verifications dominate, but parallelized across cores they are well under ~1.5 h; once sigs are fast the bottleneck shifts to parsing 650 GB, hashing, and UTXO apply (~2-4 h). 64 GB RAM keeps the whole UTXO set resident, removing the I/O bottleneck that usually dominates native IBD.

Speed ceiling, honest: a browser node is ~1.3-2x slower than native and cannot beat it at equal work. Native libsecp uses MULX/ADX and SHA-NI instructions that WASM cannot reach; the only future lever that closes most of the gap is the WASM wide-arithmetic proposal (add-with-carry, 128-bit multiply), which would take secp toward ~1.1-1.3x. We win by doing less work (assumevalid/assumeutxo), not by out-running C.

Milestones

  • M0, benchmark: WASM-SIMD secp256k1 verify throughput in workers on real blocks. Confirms the whole model. Nothing else is worth building until this number is real.
  • M1, headers in OPFS: headers-first sync to tip via the local shim, validate all PoW + difficulty, persist to OPFS. Honest, impressive, sidesteps signatures.
  • M2, light wallet: BIP158 filter scan backwards from tip, build a wallet's UTXO/history, verify inclusion via SPV. Usable wallet.
  • M3, assumeutxo: load a snapshot, full block validation forward from it (sigs via WASM workers, UTXO in RAM). Usable immediately, validating new blocks live.
  • M4, backfill: validate genesis to snapshot in the background; flip the UI from "trusted snapshot" to "fully validated".
  • M5, durability/PWA: persistent storage, installable, survives restarts, resumes.

Open questions / unknowns

  • OPFS at ~650 GB is the least battle-tested part: quotas, eviction, fragmentation, behaviour at scale.
  • Real WASM-secp throughput in workers (the model's biggest uncertainty).
  • TCP shim: peer discovery, eclipse resistance, trust framing when blocks come from few sources.
  • Tab/worker lifetime across a multi-hour sync.
  • Timeline of the WASM wide-arithmetic proposal in V8.

Non-goals (at least initially)

  • Archival node (we prune; we do not serve historical blocks to others).
  • Mining, mempool/relay policy.
  • Matching native speed (the honest target is a small constant factor off native, in a tab).

Honest headline: "usable in minutes via assumeutxo, fully validated from genesis in an afternoon, in a browser tab." Not "instant full node," not "replaces Bitcoin Core."

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions