Prevent AI session listing from exhausting Electron memory - #4592
Draft
shaunandrews wants to merge 3 commits into
Draft
Prevent AI session listing from exhausting Electron memory#4592shaunandrews wants to merge 3 commits into
shaunandrews wants to merge 3 commits into
Conversation
Fixes from review of the streamed-summary work: - New shared jsonl.ts line reader used by listing, loading, and migration: one splitting rule (LF/CRLF/CR) everywhere, streamed without holding the file in memory, and a max-line-length guard so a corrupted newline-free file is skipped instead of crashing the main process with an uncatchable RangeError past V8's string cap. - Migration decides everything from one content snapshot and re-detects the format after the probe, so two concurrent migrations can no longer parse pi output as legacy and rewrite a transcript down to a lone header; tmp files are uniquely named and cleaned up on failure. CR-delimited legacy files now migrate instead of losing every event. - Summary cache stores the pre-read stat, so a write landing mid-read costs one re-read instead of serving a stale summary until the file changes again. Deletes evict their cache entry immediately. - In-flight listing dedup is generation-gated: a caller that writes and then lists never joins a scan that started before its write (read-your-writes, as on trunk). - Cold listings scan through a small bounded pool instead of fully serialized, and drop the redundant re-stat and separate migration probe (one stat + one open per file in the common case). - Entries that parse to null/primitives no longer inflate eventCount or produce ghost summaries; firstPrompt is truncated at summary build so the process-lifetime cache stays bounded. - PI_SESSION_VERSION / PI_SESSION_CWD live in migration.ts only. - Tests for the new line reader, cache refresh/eviction, CR handling, concurrent migration, and the summary fields the fixture never exercised (endReason, site selection, environment). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issues
Problem
Refreshing AI task history loads every transcript in the archive into the Electron main process at once. With a large history (big tool results, image payloads), this exhausts the heap and kills Studio right after an AI task completes. The original report: a 651 MB archive crashed Electron near 1.2 GB RSS.
How this fixes it
Opening a task still loads its full conversation, unchanged.
How AI was used in this PR
AI traced the heap exhaustion to the session-listing path, implemented the streaming reader and cache, reviewed the result, and added test coverage.
Testing Instructions
Verified locally:
npm run typecheckclean,npx vitest run packages/common— 690 tests passing. The originally reported 651 MB archive now lists in ~1s at ~295 MB peak RSS (previously crashed near 1.2 GB).Pre-merge Checklist