Skip to content

fix(platform): reconcile confirmed UTXO height after wallet restart#4199

Merged
lklimek merged 5 commits into
feat/platform-wallet-storage-rehydrationfrom
fix/4178-utxo-height-storage-reconcile
Jul 24, 2026
Merged

fix(platform): reconcile confirmed UTXO height after wallet restart#4199
lklimek merged 5 commits into
feat/platform-wallet-storage-rehydrationfrom
fix/4178-utxo-height-storage-reconcile

Conversation

@Claudius-Maginificent

@Claudius-Maginificent Claudius-Maginificent commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

TL;DR: Fixes identity/asset-lock funding failing with "No UTXOs available" after a wallet restart, by making UTXO confirmation height single-sourced instead of duplicated and prone to going stale.

User story

As a Dash Platform wallet user, I want a Core-funded UTXO that confirmed while the wallet was running to still be recognized as confirmed after I restart the app, so that topping up an identity or funding an asset lock doesn't spuriously fail.

Scenario

Base flow

A wallet detects an incoming transaction while it's still in the mempool, tracks it, and later sees that same transaction confirm in a block — all while the app keeps running. Separately, a UTXO can also arrive already confirmed with no transaction record of its own (a legal apply() input, e.g. during rehydration).

Actual behavior

On the very next app restart, a UTXO that confirmed live — or one that was inserted already confirmed but recordless — reloads from disk as if it were still unconfirmed, even though it's genuinely confirmed on-chain, because its confirmation status was stored in more than one place that didn't reliably stay in sync. Anything requiring a final (confirmed or instant-locked) UTXO, such as an identity top-up or an asset-lock funding, then fails with "No UTXOs available" for funds that are, in fact, spendable. See #4178.

Expected behavior

A UTXO's confirmation status survives a restart intact, sourced from exactly one place, so it can never drift out of sync with reality again — regardless of whether it arrived with a full transaction record or not.

Detailed discussion

What was done

Stacked on #3968 (feat/platform-wallet-storage-rehydration) — this closes a gap in that PR's own rehydration path.

core_transactions.height is the single authority for a UTXO's confirmation height. core_utxos.height is removed entirely (migration V009) so the fact can no longer be stored twice. To cover every legal input shape — including a UTXO that arrives with no matching transaction record — core_transactions.record_blob is nullable, so the table can also hold height-only rows: apply() upserts one per new_utxos entry (Some(height) iff confirmed, NULL otherwise), and that upsert only ever touches rows where record_blob IS NULL — a real transaction record always wins and is never overwritten by a height-only placeholder. load_state() / get_tx_record() read height from every row but only surface a TransactionRecord for blob-bearing rows, and cross-check each decoded record's txid/height against its own typed columns (a new WalletStorageError::CoreTransactionEntryMismatch on disagreement) rather than trusting the blob unconditionally.

rs-platform-wallet's core_bridge.rs intentionally still routes confirmation-only updates through records rather than new_utxos (by design — a confirmation doesn't change UTXO topology); that file is untouched and out of scope here.

Migration V009 backfills any pre-existing recordless confirmed UTXO's height into a height-only core_transactions row before dropping core_utxos.height, so upgrading an already-populated store can't silently lose a confirmation. Height-only writes are monotonic-max (an out-of-order or unconfirmed new_utxos update can never regress an already-known placeholder height; a real transaction record always overrides a placeholder regardless of height, and a suppressed placeholder write is logged naming the txid). A typed-column/blob disagreement on core_transactions is treated as reconstructible: the blob is authoritative, the disagreement is logged, and the typed columns are best-effort repaired, rather than aborting the whole wallet-store load. rs-platform-wallet's poll loops (asset_lock::sync::proof, identity::network::payments::reconcile_sent_payments) that consume this storage layer now distinguish transient from permanent persistence errors — a permanent error is surfaced instead of retried forever.

A schema guard test (tc031_confirmation_height_is_single_sourced_in_core_transactions) asserts core_utxos.height is gone, core_transactions.height exists, and core_transactions.record_blob is nullable, so this bug class can't silently resurface.

Related: #4178.

Testing

  • cargo test -p platform-wallet-storage --all-features --locked — 667 passed / 0 failed, including rt2_nonzero_balance_survives_reopen, load_state_restores_confirmed_recordless_utxo_height, transaction_record_always_overrides_height_only_placeholder, load_state_reconciles_utxo_height_from_confirmed_transaction_record, load_state_defaults_utxo_without_transaction_record_to_unconfirmed, height_only_placeholder_does_not_regress, v009_backfills_recordless_confirmed_utxo_height, typed txid/height mismatch soft-repair tests, and the schema guard.
  • cargo clippy -p platform-wallet-storage --all-features --locked -- --no-deps -D warnings — clean.
  • cargo fmt -p platform-wallet-storage -- --check — clean.
  • cargo test -p platform-wallet --all-features --locked — 647 passed / 0 failed, including record_or_persister_or_log_retries_transient_backend_errors, record_or_persister_or_log_surfaces_permanent_backend_errors, reconcile_sent_payments_confirms_from_persisted_record.
  • cargo clippy -p platform-wallet --all-features --locked -- --no-deps -D warnings — clean.
  • cargo fmt -p platform-wallet -- --check — clean.

Breaking changes

None outside this unreleased, unmerged branch — core_utxos.height never shipped in any release. The schema change lands via an additive migration (V009) with a golden schema-freeze fingerprint bump, consistent with this crate's existing migration discipline.

Checklist

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

Prior work

Attribution

🤖 Co-authored by Claudius the Magnificent AI Agent

lklimek and others added 2 commits July 22, 2026 08:45
…re_transactions on load

core_transactions contains authoritative confirmation metadata, while confirmation-only transaction record updates do not rewrite core_utxos. Reconcile matching UTXO heights from the transaction table during load, retaining the core_utxos fallback when no transaction record exists.

Add regression coverage for mempool persistence followed by a record-only confirmation update. Related: #4178.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Codex Sol <noreply@openai.com>
…derive solely from core_transactions

core_utxos.height duplicated a fact core_transactions.height already owns
authoritatively, and only the latter was ever kept fresh on a
confirmation-only update — the exact staleness that caused #4178. Removing
the column (V007) makes core_transactions the single source for UTXO
confirmation height: load_state() now derives it unconditionally from the
transaction record, defaulting to unconfirmed when no matching record
exists (a UTXO-only changeset is legal via apply(), so this path is real,
not defensive dead code — covered by a new regression test). The write
path (UPSERT_UTXO_SQL, execute_upsert_utxo) no longer touches height at
all, and list_unspent_utxos/UnspentRow drop the field along with it.

A schema guard (tc030_core_utxos_height_column_removed) asserts the column
is actually gone via PRAGMA table_info, so this bug class can't silently
resurface. Golden schema-freeze fingerprints bumped deliberately for the
new migration, per this crate's own pinning-test convention.

Related: #4178.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Codex Sol <noreply@openai.com>
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: ccc8ea01-f683-41b1-9b3e-e1eb03535edd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/4178-utxo-height-storage-reconcile

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…ydration' into fix/4178-utxo-height-storage-reconcile

# Conflicts:
#	packages/rs-platform-wallet-storage/SCHEMA.md
#	packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs
#	packages/rs-platform-wallet-storage/tests/sqlite_blob_size_gate_on_load.rs
#	packages/rs-platform-wallet-storage/tests/sqlite_foreign_keys.rs
#	packages/rs-platform-wallet-storage/tests/sqlite_migrations.rs
#	packages/rs-platform-wallet-storage/tests/sqlite_money_column_overflow_on_read.rs
#	packages/rs-platform-wallet-storage/tests/sqlite_schema_pinning.rs
@lklimek
lklimek marked this pull request as ready for review July 23, 2026 11:29
@thepastaclaw

thepastaclaw commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

🕓 Ready for review — 5 ahead in queue (commit 9c27e3d)
Queue position: 6/11 · 2 reviews active
ETA: start ~14:42 UTC · complete ~15:05 UTC (median 23m across 30 recent reviews; 2 slots)
Queued 2h 52m ago · Last checked: 2026-07-24 13:50 UTC

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preliminary review — Codex only

The transaction-backed reconciliation correctly fixes the stale mempool-to-confirmed path when a matching transaction record exists. However, the new schema and loader silently downgrade confirmed UTXO-only changesets and migrated recordless UTXOs to unconfirmed, contradicting an existing all-features restart test and excluding those funds from final-input transaction building. The new authoritative transaction columns should also be cross-checked against their serialized records.

Validated blockers were found in the Codex precheck. Sonnet is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

  • Codex reviewers: gpt-5.6-sol — general (completed), gpt-5.6-sol — rust-quality (completed)
  • Verifier: gpt-5.6-sol — verifier
  • Sonnet: not run (deferred by blocker gate)

🔴 1 blocking | 🟡 1 suggestion(s)

🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs`:
- [BLOCKING] packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs:340-354: Recordless confirmed UTXOs lose confirmation across restart
  The single-source path fixes stale heights when `core_transactions` contains the matching record, but `apply()` still accepts a `CoreChangeSet` containing only `new_utxos`. The new writer no longer persists `utxo.height`, and these lines have no fallback when the transaction map lacks the txid, so a confirmed height-100 UTXO reloads with height 0 and `is_confirmed = false`. V009 also drops the previous fallback height from existing recordless rows. The unchanged `rt2_nonzero_balance_survives_reopen` integration test constructs exactly this legal input and, with `rehydration-apply` enabled by `--all-features`, expects the confirmed balance to remain 1,234,500; this implementation produces zero confirmed balance. This is functional, not merely cosmetic: key-wallet's transaction builder removes such UTXOs when `require_final_inputs` is enabled. Preserve confirmation in an authoritative normalized structure, retain a safe fallback, or reject and migrate/backfill confirmed UTXO-only state before removing the column.
- [SUGGESTION] packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs:297-315: Validate authoritative transaction metadata against record_blob
  The confirmation map is built from the typed `txid` and `height` columns, while the later transaction-record query decodes only `record_blob` and never compares the decoded record's txid or block height with those columns. A validly encoded but semantically inconsistent row can therefore mark a UTXO for the typed txid as confirmed while returning a transaction record for another txid or context. Since this PR makes these columns authoritative for UTXO confirmation and neighboring schema readers fail hard on typed-column/blob disagreement, decode and cross-check the corresponding `TransactionRecord` before accepting its typed confirmation metadata.

Comment thread packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs
Comment thread packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs Outdated
lklimek and others added 2 commits July 24, 2026 08:43
…ion height (#4199)

apply() legally accepts a CoreChangeSet with only new_utxos and no
matching transaction record, but core_utxos.height was dropped by
migration V009 with no fallback - a confirmed UTXO with no
core_transactions row reloaded as unconfirmed with height 0, silently
zeroing balance across a restart (reproducing the exact "No UTXOs
available" symptom issue #4178 fixed via a different path). Confirmed
via the existing rt2_nonzero_balance_survives_reopen test, which
failed before this change.

core_transactions.record_blob is now nullable so the table can hold
height-only rows: apply() upserts a height-only row for every
new_utxos entry (Some(height) iff confirmed, else NULL), and that
upsert only ever touches rows where record_blob IS NULL - a real
transaction record always wins and is never overwritten by a
height-only placeholder. load_state()/get_tx_record() read height
from every row but only emit a TransactionRecord for blob-bearing
rows, and now cross-check each decoded record's txid/height against
its typed columns (WalletStorageError::CoreTransactionEntryMismatch
on disagreement) instead of trusting the blob unconditionally.

V009 (unreleased/unmerged, so reshaped in place rather than layered)
renames to V009__single_source_core_confirmation_height and
transactionally rebuilds core_transactions with a nullable
record_blob instead of merely dropping core_utxos.height.

TC031 (tests/sqlite_migrations.rs) is renamed/strengthened to assert
the new invariant: core_utxos.height absent, core_transactions.height
present, core_transactions.record_blob nullable.

Full crate test/clippy/fmt suite verified green (see PR #4199 for the
run details). Addresses both thepastaclaw findings on PR #4199
(discussion_r3637872696 blocking, discussion_r3637872699 suggestion).

Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Backfill legacy recordless confirmation heights, keep blob records authoritative during soft repair, and make placeholder height updates monotonic. Classify poll-loop persistence failures and pin migration, fixture, and schema behavior with regressions.

Co-Authored-By: Codex <noreply@openai.com>
@lklimek
lklimek merged commit e9c9b74 into feat/platform-wallet-storage-rehydration Jul 24, 2026
5 checks passed
@lklimek
lklimek deleted the fix/4178-utxo-height-storage-reconcile branch July 24, 2026 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants