persistit: revalidate pending index insertions after the tree claim is dropped mid-store (bug 1017957 residual)

[vharseko]

Fixes #274.

Root cause (verified)

When Exchange#storeInternal has committed a split, the key-pointer pair for the new right sibling is inserted at the next level by a later iteration of the store loop. If that iteration throws a RetryException — in non-MVCC use this only happens on tree height growth with a contended claim upgrade — the handler releases the tree claim entirely before re-acquiring it. In that window a concurrent covering removeKeyRange can take the writer claim, unlink the just-split page onto a garbage chain, and retype it PAGE_TYPE_GARBAGE. The resumed iteration then inserted the now-stale pointer blindly, planting a dangling index entry to a garbage page.

This matches the CI evidence exactly: PAGE_TYPE_DATA is 1 and PAGE_TYPE_INDEX_MIN is 2, so the observed invalid page type 30: should be 2 means searches were descending through a root entry into a garbage-collected level-1 index page — the page split off the root whose pending root entry was planted after the page had already been unlinked. The volume stays consistent at rest once later removes delete the entry, which is why IntegrityCheck reported 0 faults. (See the correction comment on #274: two residual-mechanism hypotheses from the original analysis did not survive verification; this one did.)

Fix

• storeInternal tracks that a committed split's key-pointer pair is still pending when a retry handler releases the tree claim, and re-validates it under the re-acquired claim at the top of the main loop: a search for the pending key at the child level must land on the pending page itself (via the B-link right-walk, since the page has no parent entry yet). If it lands elsewhere, the page — and its keys — are gone, and the deferred insertion is abandoned.
• fixIndexHole is additionally hardened against stale CleanupIndexHole actions whose page has been garbage collected and reused as a live indexed page of the same tree at the same level (ABA): both existing guards pass legitimately against the new incarnation, and the action used to perform a parent-level store that is a harmless same-key replace only while every parent entry key equals its page's first key. The repair now runs only when the parent level really has no entry for the page, dropping stale actions outright.

Tests

New Bug1017957ResidualTest covers both deterministically (the original Bug1017957Test#induceCorruptionByStress reproduces the race only probabilistically under a 10-second stress):

• pendingSplitPointerRevalidatedAfterTreeClaimDrop uses a new STORE_PENDING_SPLIT ThreadSequencer schedule to park the writer inside the retry window with the root split committed, runs a covering remove that garbage-collects the pending page, and asserts no index entry points to it, IntegrityCheck is clean, and a full traversal does not throw. Without the fix it fails with No index entry may point to the garbage page expected:<0> but was:<1>.
• staleIndexHoleActionOnReusedPageIsDropped unlinks a page, drives its reuse through the garbage chain, then performs a stale CleanupIndexHole against it and asserts the action stores nothing (tree store counter unchanged) and the page keeps exactly one parent entry. Without the fix it fails with expected:<73> but was:<74>.

Verified: both new tests fail with the fixes reverted and pass with them; Bug1017957Test stress passes with Total errors 0; full persistit/core suite: 569 tests, 0 failures, 0 errors (5 pre-existing skips).