perf: enrichment cold start 8-10s + 5s TTL causes perpetual full rescans — switch to mtime-keyed per-file cache

[grimmerk]

Symptoms (measured on the 1.0.83 build, 2026-07-13)

Fresh app launch → Sessions tab: ~2s until the last-AI-message lines (◀) appear, 8–10s until custom titles appear. Values persist afterwards (renderer merges results), but the cost repeats more than it should (see TTL flaw below).

Two related defects existed before this investigation and were already fixed on the PR #132 branch (d07c385):

• Random title/branch dropout (pre-existing, seen in old builds too): ~400 concurrent exec greps (100 sessions × 4) starved the biggest transcripts past the silent 3s timeout (errors resolve to '') → the busiest session's title/branch vanished at random. Fixed by batching (10/batch) + 5s timeout.
• Branch read from tail -n 5: an active session's tail is often tool output with no gitBranch field (measured on a live 26MB session: tail -5 hit 0, tail -20 hit 12). Fixed by widening to 50 lines.

Remaining structural problems

1. Cold cost: every scan runs 3 full-file greps per session (custom-title, ai-title, pr-link) over up to ~100 transcripts (~894MB corpus on the reference machine). That's the 8–10s.
2. TTL flaw (cubic finding on PR feat(sessions): Batch 1 — full-prompt search, match snippets, junk fold, highlight fix #132, discussion r3566660448): titlesCacheTimestamp is set to the value captured at function entry. When the scan itself takes longer than the 5s CACHE_TTL_MS, the just-written cache is already stale → the next caller starts another full scan — perpetual rescans while the popup is in use. There is also no in-flight dedup, so concurrent callers (top-100 pass, VS Code pass, deep-search lazy enrichment) each start independent full scans.
3. Per-call result maps are rebuilt from scratch, so a caller passing a different session set (e.g. deep-search appended matches) can briefly observe holes within a TTL window.

Fix direction (being implemented on the PR #132 branch)

mtime-keyed per-file incremental cache:

• Module-level accumulator maps (titles/branches/prLinks) + per-file {mtimeMs, size} scan state.
• On each call: stat every requested transcript (~1–2ms total for 100 files); re-grep only new or changed files; serialize scans through a promise queue (in-flight dedup that stays correct for different session sets); drop the wall-clock TTL entirely — the stat check is the freshness test.
• First-ever launch still pays one cold scan; afterwards only actively-growing transcripts are re-read.

Possible follow-ups if still needed: single-pass grep (all 4 patterns in one scan), per-batch streaming of results to the renderer (top rows light up first), and ultimately the Batch-2 FTS index (docs/session-finding-plan.md §5.2) subsumes this entirely (it tracks per-file byte offsets anyway).

"Would rewriting in Rust help?"

No — the bottleneck is disk IO + process spawning + re-scanning unchanged bytes, not per-byte scan speed. A Rust scanner (e.g. ripgrep) would speed up the scanning fraction, but the durable win is reading less (mtime cache / index). Same conclusion as the survey in docs/session-finding-plan.md §2: none of the ecosystem tools rely on exotic engines.

Cross-refs

• perf: session tab switching performance regression (PR #103) #106 — session tab switching perf (different scope: renderer tab switch)
• perf: terminal/IDE badge appears 0.5-1s late — show last-known badge first (SWR) #133 — terminal badge SWR (same "show cached first" philosophy)
• search: only the ~100 loaded sessions are searchable; main-side searchClaudeSessions (500-pool) is dead code #131 / PR feat(sessions): Batch 1 — full-prompt search, match snippets, junk fold, highlight fix #132 — the search work that surfaced all of this

🤖 On behalf of @grimmerk — generated with Claude Code

[grimmerk]

Status after the mtime-keyed cache landed (PR #132, f8cfa43)

Measured on the packaged build (2026-07-13): cold start to custom titles went 8–10s → ~4s, and that cost is now once per app launch — subsequent refreshes stat ~100 files (~1–2ms) and re-grep only the transcripts that actually changed (typically the 1–5 active sessions), with results accumulating (no blink-out).

Remaining option: persist the cache to disk

The ~4s that's left is the first full scan of every transcript after each app launch. To eliminate it too:

• Persist {sessionId → {mtimeMs, size, title, branch, prNumber, prUrl}} to a small JSON (or the future SQLite DB) under ~/.config/codev/, loaded at startup.
• On launch, the stat pass then skips every unchanged file — first paint gets titles in the time of a stat sweep + a handful of greps for files that changed while the app was closed.
• Invalidation stays trivial: same mtime+size key; invalidateSessionCache() clears both the in-memory maps and the persisted file.

Alternatively, just wait for the Batch-2 FTS index (docs/session-finding-plan.md §5.2) — it persists per-file scan offsets anyway and would subsume this entirely. The JSON persistence is only worth doing if the 4s bothers us before FTS lands.

🤖 On behalf of @grimmerk — generated with Claude Code