Skip to content

fix: recover a corrupt refresh lock instead of freezing ingestion - #937

Draft
ozymandiashh wants to merge 1 commit into
getagentseal:feat/core-extractionfrom
ozymandiashh:fix/port-misc
Draft

fix: recover a corrupt refresh lock instead of freezing ingestion#937
ozymandiashh wants to merge 1 commit into
getagentseal:feat/core-extractionfrom
ozymandiashh:fix/port-misc

Conversation

@ozymandiashh

Copy link
Copy Markdown
Collaborator

Ports five upstream fixes. The first is the one that matters.

A corrupt lock froze ingestion permanently

A lock file whose body never parses — the zero-byte leftover of a crash between open and the body write, or a heartbeat truncated by a full disk — was classified 'unavailable'. Every later refresh then took the read-only path, permanently. Nothing on the machine repairs that file. New sessions stopped being ingested, the menubar served the pre-crash snapshot forever, and the only remedy was deleting the lock by hand.

Corruption is now its own class, recovered through the unmodified staleness gate: waited out while fresh, because it may belong to a live owner whose heartbeat is about to repair it, then taken over once its mtime ages past the stale window. That bounds the freeze to one window instead of forever. 'unavailable' is reserved for locks that genuinely cannot be read.

The takeover stability check now compares raw bytes as well as token and mtime — on filesystems with coarse mtime granularity a live owner's heartbeat can rewrite a body without moving mtime, and token equality alone could not distinguish "unchanged" from "rewritten".

Both invariants the lock exists for are preserved and now tested directly: a corrupt body is never stamped over, so a displaced writer cannot resurrect itself and delete a live successor's lock; and staleness is never waived for corruption, so two processes cannot hold it at once.

The other half of the freeze was in the daily cache

A timed-out refresh serves the prior snapshot. When anything changed underneath, that snapshot is partial — and finalizing history off it advanced the watermark past days the parse never covered. The gap scan starts after the watermark, so the hole became invisible to it forever and empty days froze into the trend chart.

The parser now reports a stale read-only serve as an incomplete hydration, and the daily cache refuses to advance its watermark or mark itself complete unless the parse behind it was complete. Caches already corrupted this way heal once: a watermark that outruns its newest populated day is pulled back so the ordinary gap parse re-derives the tail, and a trust stamp stops a legitimately idle tail being re-derived on every launch.

Two satellite fixes

  • Context budget double-counted when the project IS the home directory — every skill and ~/.claude/CLAUDE.md counted twice, roughly doubling the estimate for dotfiles repos and anyone running from ~. Dedup is now by resolved path, so a symlinked home is caught too.
  • The optimize result-cache key was a projection of project count plus api-call sum, so any two datasets agreeing on those collided and served stale findings within the TTL. It now folds in cost, savings and proxied cost.

On that last one, precisely: the key is still a projection, of five aggregates now. Datasets agreeing on all five — same totals, different per-model distribution — can still collide, and the detectors keying off token ratios and tool counts would differ. The 60-second TTL bounds the damage. The comment says that rather than claiming the collision is closed, which an earlier draft did.

Verification

Source stashed and suites re-run against the unfixed code to confirm each test bites: 7/8 corrupt-lock tests red, 2/3 parser completeness red, 4/7 daily-cache degraded red, 3/3 optimize cacheKey red, 1/2 context-budget red — including the end-to-end test that proves the unfixed code freezes ingestion.

A lock file whose body never parses — the zero-byte leftover of a crash between
open and the body write, or a heartbeat truncated by a full disk — was
classified as 'unavailable'. Every later refresh then took the read-only path,
permanently. Nothing on the machine repairs that file, so new sessions stopped
being ingested, the menubar served the snapshot from before the crash forever,
and the only remedy was deleting the lock by hand.

Corruption is now its own class. A body that fails to parse is recovered
through the unmodified staleness gate, exactly like an abandoned lock: waited
out while fresh, because it may belong to a live owner whose heartbeat is about
to repair it, then taken over once its mtime ages past the stale window. That
bounds the freeze to one stale window instead of forever. 'unavailable' is
reserved for locks that genuinely cannot be read.

The takeover stability check now compares the raw bytes as well as token and
mtime: on filesystems with coarse mtime granularity a live owner's heartbeat
can rewrite a body without moving mtime, and token equality alone could not
tell "unchanged" from "rewritten".

The other half of the freeze was in the daily cache. A timed-out refresh serves
the prior snapshot; when anything changed underneath, that snapshot is partial,
and finalizing history off it advanced the watermark past days the parse never
covered. Since the gap scan starts after the watermark, the hole became
invisible to it forever and empty days froze into the trend. The parser now
reports a stale read-only serve as an incomplete hydration, and the daily cache
refuses to advance its watermark or mark itself complete unless the parse
behind it was complete. Caches already corrupted this way heal once: a
watermark that outruns its newest populated day is pulled back so the ordinary
gap parse re-derives the tail, and a trust stamp keeps a legitimately idle tail
from being re-derived on every launch.

Two satellite fixes ride along. The context budget counted every skill and the
home CLAUDE.md twice when the scanned project IS the home directory; dedup is
now by resolved path, so a symlinked home is caught too. And the optimize
result-cache key was a projection of project count and api-call sum, so any two
datasets agreeing on those collided and served stale findings within the TTL;
it now folds in cost, savings and proxied cost.

Being accurate about that last one: the key is still a projection of five
aggregates, so datasets agreeing on all five — same totals, different per-model
distribution — can still collide, and the detectors that key off token ratios
and tool counts would differ. The 60-second TTL bounds it. The comment says so
rather than claiming the collision is closed.
@ozymandiashh
ozymandiashh marked this pull request as draft August 5, 2026 10:04
@ozymandiashh

Copy link
Copy Markdown
Collaborator Author

Converted to draft: this was opened with only a self-review by the model that wrote it, which is not the review bar this batch is held to. Independent review by two other reviewers is running now; I will mark it ready once both have passed and any findings are applied. Apologies for the noise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant