Recover from recurring LMDB corruption and unhandled commit rejections - #636
Open
gemammercado wants to merge 3 commits into
Open
Recover from recurring LMDB corruption and unhandled commit rejections#636gemammercado wants to merge 3 commits into
gemammercado wants to merge 3 commits into
Conversation
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/vitestThe overall coverage in the Show a code coverage summary of the most impacted files.
Updated |
satyakigh
requested changes
Jul 15, 2026
satyakigh
left a comment
Collaborator
There was a problem hiding this comment.
Lets hold off on this change since #615 has not been deployed yet. This might be a non issue.
Some other issues
- Concurrent writes wipe the database on the first corruption - irst corruption → cheap reopen (preserves data); recurring corruption within 30s → deleteAndRecreate() (wipes data). But when two or more writes are in flight in the same failed commit batch, they each carry the same commitError promise. The PR attaches a separate catch per operation, so a single underlying corruption event drives handleError N times back-to-back in the same microtask window. The second call sees now - lastCorruptionAt < 30s and escalates straight to deleteAndRecreate()
- The new .catch is attached only around the first await fn() (line 55). The retry return await fn() (line 60) is outside that guard. When the retry also fails with a commitError (persistent corruption — precisely the scenario the PR targets), that second promise has no handler and surfaces as an unhandled rejection with a pure-lmdb stack — the original symptom.
- MDB_BAD_TXN/MDB_CURSOR_FULL are frequently transient (aborted/nested txn, cursor exhaustion) and don't imply on-disk corruption. the new logic, a recurring transient MDB_BAD_TXN within 30s now triggers a full data wipe
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
LMDB errors are a top source of uncaught exceptions / unhandled promise rejections from the language server. Two distinct paths were involved:
On-disk corruption is never actually cleared at runtime.
handleErrorroutes corruption errors (MDB_CORRUPTED,MDB_PAGE_NOTFOUND,MDB_BAD_VALSIZE,MDB_CURSOR_FULL,MDB_BAD_TXN) torecoverFromError, which callsreopenEnv()first. Re-opening re-maps the same on-disk files and does not throw for page-level corruption (open/openDBare lazy), so recovery reports success without ever escalating todeleteAndRecreate(). The retried op hits the same corrupt page and fails again, and every subsequent operation in that process keeps failing — a loop that only ends when the editor restarts.lmdb-js commit failures reject a promise nobody awaits. On a failed commit, lmdb-js rejects each per-op promise with
Error('Commit failed (see commitError for details)')and separately rejects an internalcommitRejectPromise(exposed only aserror.commitError) with the real underlying error. Nothing awaitscommitError, so it surfaces as an unhandled rejection with a stack entirely insidelmdb/dist/index.cjsand no application frames.Fix
LMDBStoreFactory.handleError: route corruption codes to a newrecoverFromCorruption()that escalates. The first corruption within a window reopens (cheap, preserves data, covers rare transient/handle-level failures); if corruption recurs withinCorruptionEscalationWindowMs, it callsdeleteAndRecreate()to clear the corrupt files so the retry and future ops succeed on a fresh env. Emitscorruption.detected/corruption.recreatecounters.LMDBStore.execAsync: when a caught error carries acommitErrorpromise, attach a handler that routes its underlying cause through the normal recovery path. This prevents the unhandled rejection and lets the real corruption code drive escalation.The escalation preserves existing behavior on the first occurrence (reopen), so transient failures still recover without data loss.
Testing
npm run build, lint, and the full LMDB unit suite pass (181 tests, including new escalation and commit-rejection tests).execAsyncconsumes thecommitErrorpromise and routes its cause to recovery.