feat(intelligence): preserve complete customer evidence#35
Conversation
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — a95490ff
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-13T09:46:43Z
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — dd0186ff
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-13T09:52:37Z
tangletools
left a comment
There was a problem hiding this comment.
🟡 Value Audit — sound-with-nits
| Verdict | sound-with-nits |
| Concerns | 2 (2 weak-concern) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 181.7s (2 bridge agents) |
| Total | 181.7s |
💰 Value — sound-with-nits
Consolidates 6 duplicated per-adapter JSONL parsers into one streaming reader, streams Codex spans row-by-row, and fixes a misleading retry metric — real memory win (1.7GB→445MB) with bit-identical output; ship.
- What it does: Adds src/jsonl.ts exporting readJsonl (async generator over createReadStream+readline, skipping malformed rows), collectJsonl (array collect), and takeJsonl (bounded first-N rows). Claude/Copilot/Factory/Pi/Qwen swap parseLines(await readFile) for collectJsonl (drops the simultaneous raw-string retention). Codex goes further: locate() uses takeJsonl(path,40) for identity instead of reading the who
- Goals it achieves: (1) Cut peak memory on large sessions — Codex session listing no longer reads whole 400MB files to inspect identity (1.66GB→97MB); parse no longer holds raw string + parsed rows simultaneously (1.7GB→445MB). (2) Make the tool-use report honest: the old retryRate-as-percent-of-all-calls printed '100% retry' when every failed call was retried; the new form reports 'N/M failed calls were followed by
- Assessment: Coherent and in the grain. The duplicated parseLines lived in 6 adapters with identical skip-malformed semantics — one shared reader is the right consolidation. The streaming design (createReadStream + readline, explicit lines.close()/input.destroy() in finally so takeJsonl's break is safe) is the standard Node pattern and correctly preserves the old tolerant skip-on-bad-row behavior. The report f
- Better / existing approach: Checked agent-eval 0.116.0: it exports parseJsonlRows(text:string) from the benchmarks subpath, but it is a STRING parser (you must readFile first, holding raw+parsed in memory — exactly what this PR eliminates) and it THROWS on the first malformed row instead of skipping. Not a substitute. No pre-existing streaming JSONL reader in this repo either (grepped src/ for createReadStream/createInterfac
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound
A focused memory fix that consolidates 6 duplicate parseLines into one shared streaming JSONL reader, streams Codex directly to spans, and corrects a misleading retry-percentage report — all wired into live call paths.
- Integration: Fully reachable: the 6 consuming adapters are registered in registry.ts:26 and driven by session-source.ts parseSession/scanSessions (exported at index.ts:19); renderPipelines is called from cli.ts:420 and improvement.ts:553. No dead surface — every helper has a caller.
- Fit with existing patterns: Matches the codebase's grain precisely. Each adapter previously had its own copy of the same read-file-then-split-then-parse routine; this PR collapses them into one shared jsonl.ts. The Codex true-streaming path is the natural extension for the one adapter with a proven 400MB problem. No competing pattern exists (session-index.ts:209 does key-counting, a different job).
- Real-world viability: Holds up: createReadStream+readline with crlfDelay:Infinity and a finally-block destroy is the canonical Node streaming pattern; takeJsonl breaks early and cleans up; malformed rows are skipped without aborting the trace. Codex parse() may read the file up to 3x in the rare case meta/model aren't in the first 40 lines (head + fallback scan + main loop), but the common case is head + single body pa
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
💰 Value Audit
🟡 Codex parse reads the file twice (bounded head + full stream) [maintenance] ``
src/adapters/codex.ts:199 reads takeJsonl(path,40) for first/meta/model, then src/adapters/codex.ts:234 re-streams the whole file via readJsonl to build spans. A single-pass design could buffer the first 40 rows inline during the building loop and defer root-span creation until meta/model are seen. In practice the head read is bounded to 40 lines so the extra I/O is negligible and the two-phase shape makes the bit-identical-output guarantee trivial to reason about — not worth changing, but a rev
🟡 adoption.ts still uses the old readFile+split JSONL pattern [maintenance] ``
src/adoption.ts:115-133 readLoopRuns does readFile(path,'utf8') then countSkillRunsJsonl(raw) splits on '\n' — the exact pattern this PR removed from the adapters. Now that readJsonl exists, readLoopRuns could stream via readJsonl for consistency. Not blocking: .evolve/skill-runs.jsonl files are tiny, and countSkillRunsJsonl takes a raw string deliberately (testable, sync aggregation). Note for a later consolidation pass, not this PR's scope.
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 4fa24975
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-13T10:06:01Z
tangletools
left a comment
There was a problem hiding this comment.
🟢 Value Audit — sound
| Verdict | sound |
| Concerns | 0 (none) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 355.8s (2 bridge agents) |
| Total | 355.8s |
💰 Value — sound
Replaces 6 copy-pasted JSONL parsers with one streaming reader, cuts peak memory on large sessions by ~75%, and fixes a misleading '100% retry' display — clean, in-grain, ship.
- What it does: Introduces src/jsonl.ts with three exports — readJsonl (streaming async generator), collectJsonl (materialize all), takeJsonl (first N rows). Removes 6 identical parseLines functions across the claude/codex/copilot/factory/pi/qwen adapters and routes them through the shared reader. Codex discovery now reads only the first 40 parsed rows (takeJsonl) instead of slurping the whole file; Codex parse s
- Goals it achieves: (1) Cut peak parse memory on large JSONL sessions — the old readFile+split retained both the raw string and the parsed array, peaking at ~1.7GB on a 400MB Codex session; the streaming reader never holds the raw text. (2) Cut session-listing memory ~17x by reading only the header lines needed for cwd/id during discovery. (3) Fix the report's '100% retry' claim — retryRate is a ratio of error calls,
- Assessment: Coherent and well-executed. The shared reader is minimal (38 lines), correctly typed as a generic generator, tolerates malformed rows (matching the old behavior), and cleans up its stream/readline in a finally block. Routing 6 adapters through it removes genuine copy-paste duplication. The Codex streaming parse is a real architectural improvement: discovery does bounded head reads, and the main lo
- Better / existing approach: none — this is the right approach. Checked @tangle-network/agent-eval (the direct dependency) for any existing JSONL streaming utility; none exists. Two other inline JSONL parsers remain in the codebase (session-index.ts:209 summarizeJsonl and adoption.ts:89 countSkillRunsJsonl), but both take raw string input rather than a path, and summarizeJsonl specifically needs to count invalid rows (which r
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound
A coherent dedup-and-stream refactor: one shared JSONL reader replaces six identical parseLines helpers, Codex streams directly into spans, and the misleading '100% retry' report now shows explicit numerators/denominators.
- Integration: Fully reachable. The new readJsonl/collectJsonl/takeJsonl in src/jsonl.ts are consumed by all 6 JSONL adapters (claude.ts:274,327; codex.ts:181,199,204,235; copilot.ts:77; factory.ts:103; pi.ts:114; qwen.ts:92), all registered in src/registry.ts:27-36 and called from cli.ts/improvement.ts/live.ts. The 4 unmigrated adapters (gemini, opencode, forge, amp) parse whole-document JSON, not line-delimite
- Fit with existing patterns: Matches the codebase grain exactly. Six copy-pasted parseLines functions (each ~10 lines, identical except the row type) collapse into one typed async generator plus two thin helpers. The interface mirrors how the rest of the code consumes things: collectJsonl returns T[] as before, takeJsonl is the bounded-head read Codex's locate/parse already needed. No competing pattern exists.
- Real-world viability: Stream cleanup is correct (try/finally in src/jsonl.ts:17-20 closes readline and destroys the stream even on early break). Malformed-row tolerance matches prior behavior. The codex parse path's triple-read (head scan -> optional meta fallback -> main loop) is bounded and the wall-time claim (2.55s unchanged) is plausible since the win is memory, not I/O. The report.ts:152 retry math (retryRate * e
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
No concerns — sound change, no better or existing approach found. ✅
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
✅ No Blockers —
|
tangletools
left a comment
There was a problem hiding this comment.
✅ Approved — 14 non-blocking findings — 4fa24975
Full multi-shot audit completed 8/8 planned shots over 14 changed files. Global verifier still owns final merge decision.
Full immutable report for this review: trace
Summary comment for this run: full summary
tangletools · 2026-07-13T10:38:54Z · immutable trace
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 8abcfd03
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-13T10:56:48Z
tangletools
left a comment
There was a problem hiding this comment.
🟡 Value Audit — sound-with-nits
| Verdict | sound-with-nits |
| Concerns | 1 (1 weak-concern) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 246.3s (2 bridge agents) |
| Total | 246.3s |
💰 Value — sound-with-nits
Adds a shared streaming JSONL reader and migrates 6 adapters off read-whole-file parsing, cutting peak memory ~4x and fixing a misleading '100% retry' report; clean and in-grain, with one deliberate behavior change worth a glance.
- What it does: Introduces src/jsonl.ts (readJsonl async generator + takeJsonl bounded reader + JsonlParseError). Migrates claude, codex, copilot, factory, pi, qwen away from readFile-then-split-then-parseLines-retain-both. Codex locate() reads only 40 parsed rows for identity discovery; Codex parse() and Claude parse() stream rows straight into spans. Claude's parseClaudeStream is refactored into createClaudeStr
- Goals it achieves: (1) Cut peak memory on large sessions by not retaining both the raw file string and the full parsed-row array — PR claims 1,695,564 KB -> 444,816 KB on a 402 MB Codex session, which is exactly what streaming line-by-line buys. (2) Speed up session listing by bounded-head discovery — 0.87 s / 1.66 GB -> 0.14 s / 97 MB. (3) Stop the misleading '100% retry' display, which arose because retryRate (a f
- Assessment: Sound and in the grain of the codebase. The shared reader is the right size (48 lines, takeJsonl composes on readJsonl), lives where every harness adapter can reach it, and replaces six near-identical private parseLines copies. The Claude refactor preserves the exported parseClaudeStream signature (still takes a readonly ClaudeEvent[]) while letting the streaming parse path reuse consumeClaudeEven
- Better / existing approach: none — this is the right approach. Searched: (a) rg for an existing generic JSONL reader in src/ (only the new src/jsonl.ts and an unrelated readline use in src/cli.ts); (b) agent-eval's exported API for a reusable stream reader (absent — its JSONL helpers are all domain-shaped around RunRecord/ReferenceReplay/Gold); (c) node_modules/@ax-llm/ax (no JSONL reader). A ~48-line local utility is correc
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound
Consolidates duplicated per-adapter JSONL parsing into one streaming reader that cuts peak memory ~4x on huge sessions, fixes a misleading retry-rate report by using the correct denominator, and upgrades agent-eval — all wired into existing call paths with no dead surface.
- Integration: Fully reachable. readJsonl/takeJsonl (src/jsonl.ts) are consumed by all six modified adapters, which are registered in src/registry.ts:27-35 and re-exported via src/index.ts. renderPipelines (src/report.ts:136) is called by src/cli.ts:420 and src/improvement.ts:553. parseClaudeStream remains exported and tested (tests/traces.test.ts:65).
- Fit with existing patterns: Matches the codebase grain exactly: replaces a verbatim-copy parseLines helper that existed in every JSONL adapter with one shared streaming reader. No competing or established alternative pattern exists in the repo.
- Real-world viability: Streaming uses canonical Node createReadStream+createInterface with proper finally cleanup; empty/missing/malformed inputs each have explicit tests (tests/jsonl.test.ts, tests/adapters.test.ts:48-67). A 100MB/128MB-heap child-process test (tests/adapters.test.ts) directly proves the memory claim. The retry-rate fix uses the correct denominator, verified against agent-eval source (retryRate = retri
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
💰 Value Audit
🟡 Malformed-line tolerance was deliberately removed — verify it holds against real Copilot files [maintenance] ``
Every prior parseLines silently skipped bad lines (src/adapters/copilot.ts had an explicit comment: 'GitHub's writer has shipped lines with raw separators — skip tolerantly'). The new readJsonl throws JsonlParseError on the first unparseable line (src/jsonl.ts:29), so one bad line now aborts the whole session parse, yielding partial spans up to that line then throwing. This is a deliberate, tested choice (commit 8abcfd0 'fail loudly while streaming JSONL'; tests/jsonl.test.ts:22 asserts the type
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
❌ Needs Work —
|
tangletools
left a comment
There was a problem hiding this comment.
❌ 1 Blocking Finding — 8abcfd03
Full multi-shot audit completed 8/8 planned shots over 15 changed files. Global verifier still owns final merge decision.
Full immutable report for this review: trace
Summary comment for this run: full summary
tangletools · 2026-07-13T11:34:56Z · immutable trace
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — d5116513
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-13T13:05:03Z
tangletools
left a comment
There was a problem hiding this comment.
🟡 Value Audit — sound-with-nits
| Verdict | sound-with-nits |
| Concerns | 4 (1 low, 3 weak-concern) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 272.3s (2 bridge agents) |
| Total | 272.3s |
💰 Value — sound-with-nits
A coherent streaming + tool-I/O refactor: parsers no longer buffer multi-hundred-MB JSONL files, tool args/results become first-class OpenInference attributes, silent error-swallowing is replaced by typed errors that don't leak raw line content; minor concerns only.
- What it does: Adds a streaming line reader (src/jsonl.ts) and a single-doc JSON reader (src/json.ts) with typed, secret-scrubbed errors; rewrites every JSONL adapter (Claude, Codex, Copilot, Factory, Pi, Qwen) to consume rows lazily via for-await; introduces src/adapters/tool-io.ts so tool input/output lands on proper OpenInference keys (input.value/output.value, mime, sha256, byte size, truncation flag) instea
- Goals it achieves: (1) Peak memory on a 402 MB Codex session drops from ~1.7 GB to ~445 MB during parse and from ~1.6 GB to ~95 MB during locate — measured, with identical 74,753-span output and SHA-256. (2) Tool I/O becomes inspectable and redactable as first-class values (OpenInference shape) rather than getting lost in the prose
contentblob. (3) Parse failures stop being silently swallowed — a malformed line o - Assessment: Good change, in the grain of the codebase. The cascade is coherent, not arbitrary scope creep: once you stream, you cannot JSON.parse(line) inline with try/catch swallowing, so you need a shared reader; the shared reader needs typed errors that don't leak raw line content; the new tool I/O attributes need their own redaction path; and the new outbound shape changes what 'same session' means for up
- Better / existing approach: Searched node_modules of @tangle-network/agent-eval 0.103 (the version before the bump): agent-eval DOES export a readJsonl (dist/index.js:8430) — but it is
readFileSync(path,'utf8').split('\n').map(JSON.parse), i.e. the exact buffered pattern causing the 1.7 GB peak. The local src/jsonl.ts is therefore NOT a duplicate; it is the streaming replacement for a substrate helper that does not stream. - Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound-with-nits
A well-wired consolidation that replaces 6 duplicate per-adapter JSONL parsers with one shared streaming reader, cutting Codex session-listing memory ~17x and fixing a misleading retry-percentage report.
- Integration: Fully reachable. readJsonl/takeJsonl (src/jsonl.ts:24,56) are consumed by 6 adapters (claude.ts:311,377; codex.ts:231,246,251,297; copilot.ts:94; factory.ts:128; pi.ts:132; qwen.ts:107). readJsonFile/isMissingPathError/isMissingJsonSource (src/json.ts) back amp/claude/opencode/gemini/factory/forge. tool-io.ts (toolIoAttributes/recordToolOutput/normalizeToolIoAttributes) is used by every adapter pl
- Fit with existing patterns: Fits the established pattern precisely. The registry already collapses many harnesses onto shared adapters; this PR applies the same consolidation to JSONL parsing — removing 6 copies of parseLines (claude.ts old lines 57-68, codex.ts old parseLines, etc.) in favor of one streaming primitive. The tool-io module unifies what was previously per-adapter JSON.stringify of tool input/output into a shar
- Real-world viability: Holds up on real paths. readJsonl uses createReadStream with a finally-destroy (jsonl.ts:51-53), handles \r\n, handles a trailing line without newline, and skips blank lines. The deliberate fail-loud JsonlParseError (jsonl.ts:3-13) replaces silent line-swallowing; the team iterated on this across the branch (commits 8abcfd0 'fail loudly', d511651 'prevent silent session data loss'), so the error p
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
🔎 Heuristic Signals
🟡 Cruft: console debug added src/cli.ts
- console.log('No sessions found to upload.')
🎯 Usefulness Audit
🟡 Library surface exports readJsonFile but not readJsonl/takeJsonl or tool-io helpers [ergonomics] ``
index.ts:18 exports readJsonFile/JsonSourceError and :19 exports JsonlParseError, but the broadly-useful readJsonl/takeJsonl streaming primitives and toolIoAttributes/recordToolOutput are NOT re-exported — even though index.ts:33-36 explicitly invites custom-adapter authors via the conversation helpers. Anyone writing a custom adapter (the documented extension path) would want the same streaming reader and tool-I/O attribute contract the built-ins use. Minor inconsistency; not a blocker since bu
💰 Value Audit
🟡 Streaming readJsonl duplicates a substrate helper by name with different semantics [maintenance] ``
@tangle-network/agent-eval already exports
readJsonl(verified in node_modules/.../@tangle-network/agent-eval/dist/index.js:8430 in v0.103; still present in the 0.116 line), but its implementation is the bufferedreadFileSync(...).split('\n').map(JSON.parse). src/jsonl.ts:24 ships a same-named async generator with streaming semantics. Two functions namedreadJsonlwith different sync/async shapes and different memory profiles is a future-confusion trap — a maintainer who imports the wrong
🟡 PR title understates scope: upload dedup + redaction rewrite is bundled with the streaming fix [proportion] ``
The title says 'stream JSONL trace analysis', but the diff also rewrites upload-state.ts (sessionHash → contentHash + new outboundHash with UploadIdentity), redact.ts (scrub input.value/output.value/status.message, not just content), and upload.ts (prepareOutbound stage, PartialUploadError, deferred uploadedAt-stamping). These are legitimate consequences of introducing the tool-io attributes — the old 4-field structural hash would miss tool-I/O changes and the old redactor pass would miss the ne
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 9515207e
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-13T14:04:44Z
tangletools
left a comment
There was a problem hiding this comment.
🟡 Value Audit — sound-with-nits
| Verdict | sound-with-nits |
| Concerns | 4 (1 low, 3 weak-concern) |
| Heuristic | 0.1s |
| Duplication | 0.0s |
| Interrogation | 305.9s (2 bridge agents) |
| Total | 306.0s |
💰 Value — sound-with-nits
Centralizes streaming JSONL + bounded corruption recovery and threads it through every adapter so isolated malformed records and non-ENOENT errors stop dropping sessions silently; well-built, in-grain, no existing equivalent to reuse.
- What it does: Adds a streaming JSONL reader (src/jsonl.ts) with two modes — strict (throws a typed JsonlParseError carrying byte offset/length/SHA-256) and recover (skips the bad record, emits a location-only receipt via onCorruption). Adds src/integrity.ts: a per-SessionRef corruption accumulator with hard bounds (≤128 receipts, ≤64 KiB serialized) that throws SessionCorruptionLimitError past the limit and sta
- Goals it achieves: (1) Stop silent session data loss — a single bad line, an unreadable subagent file, or a permission error no longer drops the whole session or is swallowed as 'no sessions'. (2) Stream multi-hundred-MB rollouts without OOM (the cited 105 MB / 415 MB regressions). (3) Make degradation visible and auditable: location+SHA-256 receipts on the root span and in the report, raw malformed bytes never reta
- Assessment: Coherent and in the grain of the codebase. The right seam: two new modules (jsonl.ts for the streaming primitive, integrity.ts for the per-session receipt accounting) consumed by every adapter through one helper (sessionJsonlOptions). Strict-vs-recover split preserves whole-file validation as an opt-in while making the default resilient — the prior parseLines catch-all (`try { JSON.parse } catch {
- Better / existing approach: none — this is the right approach. Searched node_modules/@tangle-network/agent-eval/dist for readJsonl|streamJsonl|parseJsonl|JsonlReader|createReadStream|onCorruption — no existing JSONL streamer. The only adjacent primitive is OtlpFileTraceStore (traces.d.ts:453–539), which pins the whole file in a Buffer for already-normalized OTLP and serves analyst search/view APIs; wrong layer for streaming
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound-with-nits
Streaming JSONL + bounded corruption recovery + structured OpenInference tool I/O land cleanly on the main locate→parse spine, reach every adapter uniformly, and correctly expand the redaction surface to match the new data capture surface.
- Integration: All new surface is reachable.
readJsonl/takeJsonl(src/jsonl.ts) replace the old one-shotcollectJsonl(now fully removed — grep returns no references) across every adapter (claude, codex, copilot, factory, pi, qwen, amp, opencode).readJsonFile/listJsonFiles/isMissingPathError/isMissingJsonSource(src/json.ts) replace rawJSON.parse(readFile(...))+ swallow-catch in amp, claude su - Fit with existing patterns: Fits the established grain precisely. The old pattern was: each adapter did
JSON.parse(await readFile(...))inside atry { ... } catch { return [] }that silently swallowed EVERYTHING — read errors, parse errors, missing dirs — indistinguishably. The PR replaces that with two typed primitives (readJsonFilefor single objects, streamingreadJsonlfor line streams) plus a single `isMissingPa - Real-world viability: The streaming + bounded-recovery design holds up under the realistic failure modes the PR targets.
readJsonl(jsonl.ts:98-139) is a true streaming generator overcreateReadStreamwith manual LF scanning and cross-chunk fragment reassembly — memory stays bounded regardless of file size (the 105MB / 415MB regression numbers in the PR body corroborate this). Corruption handling is isolated per-li - Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
🔎 Heuristic Signals
🟡 Cruft: console debug added src/cli.ts
- console.log('No sessions found to upload.')
🎯 Usefulness Audit
🟡 strict corruption mode is SDK-only; no CLI flag [ergonomics] ``
ParseOptions.corruptionMode: 'strict'is threaded fromScanOptionsthroughscanSessions(session-source.ts:106) intosessionJsonlOptions(integrity.ts:78), but the CLI (src/cli.ts) never sets it — every CLI invocation runs in the defaultrecovermode, and there is no--strictflag inparseArgs. The PR body explicitly frames recover-as-default + strict-for-validation as the intended contract, and recover IS the right default for operator tooling, so this is not a blocker. But an ope
💰 Value Audit
🟡 Corruption receipt dedup keyed by SessionRef object identity [maintenance] ``
src/integrity.ts:15 keys the receipt-seen cache on a WeakMap<SessionRef, ReceiptState>. That is fine for the single parseSession() pass (src/session-source.ts:28 calls stampSessionIntegrity once per ref), but a future caller that constructs two SessionRef objects for the same underlying file would lose dedup and re-record receipts against both. Not a blocker for the current single-parse spine; worth a one-line comment at the WeakMap declaring the object-identity assumption so a later caller does
🟡 stampSessionIntegrity mutates receipts in place after trace id is known [maintenance] ``
src/integrity.ts:101–114 overwrites
harnessandsessionIdon each corruption receipt after the root span's trace_id is resolved, depending on parse having completed before stamping. The ordering is correct today because parseSession() (src/session-source.ts:26–28) calls adapter.parse → stampSessionIntegrity in that order. The coupling is implicit; a future adapter that stamped integrity mid-parse would get half-enriched receipts. A one-line note in the JSDoc that stamp must run post-parse wo
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — f84d330a
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-13T14:48:09Z
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — f84d330a
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-13T14:48:11Z
tangletools
left a comment
There was a problem hiding this comment.
🟡 Value Audit — sound-with-nits
| Verdict | sound-with-nits |
| Concerns | 2 (1 low, 1 weak-concern) |
| Heuristic | 0.1s |
| Duplication | 0.0s |
| Interrogation | 456.8s (2 bridge agents) |
| Total | 456.9s |
💰 Value — sound-with-nits
A coherent, well-architected shift from read-everything-then-skip-the-bad-line to streaming JSONL with per-record corruption receipts + OpenInference tool I/O; one minor DRY nit inside the change itself.
- What it does: Replaces every adapter's 'readFile → JSON.parse the whole file → swallow errors' pattern with a single streaming
readJsonl(src/jsonl.ts) that yields valid records and emits one immutablesource.corruption.receiptchild span (location + SHA-256, never raw bytes) per malformed line, with strict mode that throws a typed JsonlParseError. AddsreadJsonFile(src/json.ts) with typed errors, an int - Goals it achieves: (1) No silent data loss — every corrupt JSONL record is accounted for as evidence rather than dropped, while every valid record still flows through. (2) Bounded memory — stream files instead of holding parsed arrays (regression test pins a 100MB file under 128MB RSS). (3) Fail loudly on unreadable sessions / zero-span parses / partial backend acceptance instead of returning []. (4) Capture tool I/
- Assessment: Sound and in the grain of the codebase. The new primitives are small, single-purpose, and exported from src/index.ts for SDK consumers. The spine stays in session-source.ts (parseSession calls stampSessionIntegrity once), adapters stay per-harness, and the substrate (agent-eval) provides the heavy lifting (contentHash, OtlpFileTraceStore, analysts). The receipt design is the right level of evidenc
- Better / existing approach: Checked for an existing JSONL reader in @tangle-network/agent-eval — it ships LockedJsonlAppender (write-side only) and canonicalJson/contentHash, but NO streaming reader. Searched src/ for createReadStream/readline/readJsonl usage — only the new src/jsonl.ts. So the reader is genuinely new capability, not a reinvention. The permissive canonicalJson in tool-io.ts differs deliberately from agent-ev
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound
Streaming JSONL readers with per-record corruption receipts, bounded tool-I/O capture, and Codex provenance are wired into the single parseSession chokepoint reached by every read path — coherent, complete, no dead surface.
- Integration: Fully reachable. parseSession (src/session-source.ts:21) is the one funnel for cli/upload/collect/live/evidence/session-index and stamps integrity on every path. All 6 JSONL adapters consume readJsonl/sessionJsonlOptions; all JSON adapters use readJsonFile; all 10 adapters plus redact+upload consume the tool-I/O helpers; Codex provenance flows to the report's SelectedSessionSource table. Exported
- Fit with existing patterns: Extends the established OTLP span+attribute model rather than competing with it: corruption evidence is emitted as source.corruption.receipt child spans (same span() primitive), tool I/O uses OpenInference input.value/output.value that runtime-store.ts:62 already consumes, and post-redaction digest recompute (normalizeToolIoAttributes) composes inside the existing redactSpans pipeline. The streami
- Real-world viability: Robust beyond the happy path: the reader handles cross-chunk line splits, CR-LF, missing trailing newline, invalid UTF-8 pre-decode, strict-vs-recover, and stream teardown in finally. Receipts are idempotent (dedup by identity tuple), upload dedup keys on post-privacy-normalization hash, PartialUploadError fails loudly on partial backend acceptance, EmptySessionError catches zero-span parses. Test
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
🔎 Heuristic Signals
🟡 Cruft: console debug added src/cli.ts
- console.log('No sessions found to upload.')
💰 Value Audit
🟡 Placeholder-trace_id → discover → remap pattern duplicated across 4 adapters [maintenance] ``
Claude (src/adapters/claude.ts:297-332), Factory (src/adapters/factory.ts:118-152), Pi (src/adapters/pi.ts:123-150), and Qwen (src/adapters/qwen.ts:98-203) each repeat the same ~10-line shape: track sourceTraceId=ref.sessionId + sourceRootId, accumulate spans under it during the stream, track firstTimestamp/lastTimestamp/sawEvent, then after the stream iterate spans to remap trace_id + parent_span_id and prepend a root. A small helper like `remapSessionTraceId(spans, sourceRootId, discoveredTrac
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
tangletools
left a comment
There was a problem hiding this comment.
🟢 Value Audit — sound
| Verdict | sound |
| Concerns | 3 (1 low, 2 weak-concern) |
| Heuristic | 0.1s |
| Duplication | 0.0s |
| Interrogation | 442.9s (2 bridge agents) |
| Total | 443.0s |
💰 Value — sound
Replaces whole-file JSON.parse + silent line-dropping with a streaming JSONL reader that recovers every valid record, fingerprints each corrupt one with byte-exact receipts, and threads bounded integrity provenance + post-redaction metadata normalization through every adapter and the upload spine —
- What it does: Replaces the prior pattern (readFile whole file → split('\n') → JSON.parse with try/catch that silently dropped malformed lines, e.g. src/adapters/copilot.ts:48-58 on origin/main) with: (1) a streaming async-generator JSONL reader (src/jsonl.ts:98-139) that yields records one at a time, never holding the source file in memory; (2) byte-exact corruption handling with two modes — 'strict' (throws Js
- Goals it achieves: (1) No silent data loss: every valid JSONL record is parsed regardless of how many corrupt records surround it; the 130-corruption regression is the proof case. (2) Bounded memory: large sessions no longer sit fully parsed in RAM. (3) Loud failures for genuinely unreadable sessions (vs. the old
catch { return [] }which silently made unreadable sessions look like zero-session cwd mismatches) — s - Assessment: Sound. The change is correctly scoped and in the grain of the codebase. The codebase already funnels every OTLP-producing path through scanSessions → parseSession (src/session-source.ts:21-34), so stampSessionIntegrity hooks the one right place. Each adapter keeps its own line-type parser but routes I/O through four shared helpers (readJsonl, readJsonFile+isMissingPathError, toolIoAttributes, reco
- Better / existing approach: none — this is the right approach. Searched the substrate for an existing equivalent before concluding:
rg 'readJsonl|streamJsonl|JsonlReader|readJsonFile' node_modules/@tangle-network/agent-eval/dist/index.d.ts→ no exports;rg 'TOOL_IO_VALUE|toolIoAttributes|input\.value.*output\.value' node_modules/@tangle-network/agent-eval/dist/traces.d.ts→ the substrate CONSUMES input.value/output.value - Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound
Streaming JSONL refactor eliminates silent data loss across all harness adapters via one shared primitive, with corruption receipts and fail-loud error paths — fully wired into the existing scan/parse/upload spine and verified to hold under corrupt-input and invalid-UTF-8 edge cases.
- Integration: Fully reachable. The new streaming primitives (readJsonl/takeJsonl) are consumed by 6 JSONL adapters; readJsonFile/listJsonFiles/isMissingPathError/isMissingJsonSource by the remaining JSON-shaped adapters (amp, opencode, gemini, forge). sessionJsonlOptions + stampSessionIntegrity ride the central parseSession spine (session-source.ts:26-28), which is consumed by 6 callers (upload, collect, sessio
- Fit with existing patterns: Consolidates rather than competes. Before, each adapter owned a whole-file readFile+JSON.parse (claude's old parseLines, amp's old JSON.parse(await readFile)). The new readJsonl async generator is the single streaming primitive all JSONL adapters share; corruption receipts project onto the existing OtlpSpan model as source.corruption.receipt child spans; tool I/O uses OpenInference input.value/out
- Real-world viability: Holds past the happy path. Verified directly: 130 interleaved corrupt records yield 130 unique SHA-256 receipts with both bracketing valid records retained; an invalid-UTF-8 line is detected and recovered around; strict mode throws at the first corrupt line (line 2); second stamping is idempotent (3→3 spans, no duplicate children); root integrity metadata is 255 bytes (<256 bound). locate/parse no
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
🔎 Heuristic Signals
🟡 Cruft: console debug added src/cli.ts
- console.log('No sessions found to upload.')
💰 Value Audit
🟡 analyze.ts ceiling rename+raise is unrelated to the silent-data-loss fix and undescribed in the PR body [maintenance] ``
src/analyze.ts:66-78 renames DETERMINISTIC_VIEW_CEILING (256MB) → GENERATED_TRACE_FILE_CEILING (512MB) AND now also sets
maxFileByteson both OtlpFileTraceStore instances. This is a 2x cap raise riding on a fix PR. The recover-mode change should make sessions SMALLER on average (corrupt lines that were silently dropped before now still parse the valid ones; the only growth is N small receipt child spans). So the cap raise isn't demanded by this PR's goal — it's a separate 'let genuinely huge s
🟡 JsonSourceError mutates the underlying SyntaxError cause in place [maintenance] ``
src/json.ts:28-34 (pathSafeParseCause) rewrites cause.message and cause.stack on the original SyntaxError rather than wrapping it. This is mildly surprising for any caller that held a reference to the original parse error, but in practice the SyntaxError is fresh from JSON.parse and only the propagated JsonSourceError is observable. Worth a comment or a wrap-in-new-Error revision eventually; not a blocker.
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
❌ Needs Work —
|
tangletools
left a comment
There was a problem hiding this comment.
❌ 2 Blocking Findings — f84d330a
Full multi-shot audit completed 8/8 planned shots over 38 changed files. Global verifier still owns final merge decision.
Full immutable report for this review: trace
Summary comment for this run: full summary
tangletools · 2026-07-13T15:42:19Z · immutable trace
✅ No Blockers —
|
Superseded by re-review — no blocking findings on latest commit.
tangletools
left a comment
There was a problem hiding this comment.
✅ Approved — 26 non-blocking findings — f84d330a
Full multi-shot audit completed 8/8 planned shots over 38 changed files. Global verifier still owns final merge decision.
Full immutable report for this review: trace
Summary comment for this run: full summary
tangletools · 2026-07-13T15:52:23Z · immutable trace
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — bbc9fdda
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-13T16:32:33Z
tangletools
left a comment
There was a problem hiding this comment.
🟢 Value Audit — sound
| Verdict | sound |
| Concerns | 3 (2 low, 1 weak-concern) |
| Heuristic | 0.1s |
| Duplication | 0.0s |
| Interrogation | 273.8s (2 bridge agents) |
| Total | 273.9s |
💰 Value — sound
Replaces whole-file session reads with streaming JSONL + corruption-receipt recovery across 6 adapters and tightens report honesty — coherent, in-grain, no existing equivalent to duplicate.
- What it does: Converts all JSONL-based harness adapters (Claude, Codex, Copilot, Factory, Pi, Qwen) from readFile+split-the-whole-file into a streaming async-generator readJsonl with two corruption modes: strict (throw on first bad record) and recover (skip the record, emit an immutable source.corruption.receipt child span carrying source path, line number, exact byte offset/length, and SHA-256 — but never the
- Goals it achieves: Eliminate four classes of silent data loss the prior code had: (1) malformed JSONL lines silently skipped via catch{}, (2) unreadable discovered sessions silently returned [], (3) zero-span parses silently filtered, (4) partial backend acceptance silently ignored. Secondary goals: make corruption auditable (location+hash receipts) without retaining raw malformed bytes in traces/uploads; make tool
- Assessment: Sound. The four new modules (jsonl.ts, json.ts, integrity.ts, tool-io.ts) are each single-purpose and consumed uniformly across the adapters that need them — the factoring matches the existing adapter-module conventions. The streaming reader is the correct Node.js primitive (createReadStream + buffer accumulation for split chunks) and the receipt design is well-considered: fingerprint-only (sha256
- Better / existing approach: Checked agent-eval 0.116.0 (npm-packed and inspected dist/index.js): its internal readJsonl is
readFileSync(path,'utf8').split('\n')— the exact whole-file antipattern this PR fixes, and it is NOT exported (no readJsonl in any .d.ts). So there is no existing substrate equivalent to reuse. The PR correctly builds the streaming+receipt infrastructure locally and correctly reuses agent-eval's expor - Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — error
usefulness agent produced no parseable value-audit JSON.
- Model: opencode/deepseek/deepseek-v4-pro
- Bridge attempts: 3
- Bridge error: opencode/zai-coding-plan/glm-5.2: bridge stream ended without value-audit content; opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content; opencode/deepseek/deepseek-v4-pro: bridge stream ended without value-audit content
🔎 Heuristic Signals
🟡 Cruft: console debug added src/cli.ts
- console.log(
🟡 Cruft: magic number added src/report.ts
lines.push(` - \`${f.toolName}\` ×${f.occurrences} with identical args across ${(f.windowMs / 1000).toFixed(1)}s`)
💰 Value Audit
🟡 Streaming JSONL + corruption-receipt pattern is a substrate-upstream candidate [maintenance] ``
The jsonl.ts streaming reader with sha256 corruption receipts solves a problem every agent-eval consumer will face when reading large session files (the PR's own proof: 420MB / 158K-line operator session at 116MB peak RSS). agent-eval's own internal readJsonl (inspected in @tangle-network/agent-eval@0.116.0 dist/index.js) still uses readFileSync+split — the antipattern. Once this pattern is proven in traces (the PR ships regression tests at tests/jsonl.test.ts:55-80 and a 130-corruption regressi
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
✅ No Blockers —
|
tangletools
left a comment
There was a problem hiding this comment.
✅ Approved — 26 non-blocking findings — bbc9fdda
Full multi-shot audit completed 8/8 planned shots over 43 changed files. Global verifier still owns final merge decision.
Full immutable report for this review: trace
Summary comment for this run: full summary
tangletools · 2026-07-13T18:58:47Z · immutable trace
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 1da788dd
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-14T04:26:21Z
|
| State | Detail |
|---|---|
| Interrupted | webhook restarted |
No review verdict was produced for this run. Trigger a fresh review on the current PR head if the PR is still open.
tangletools · #35 · model: kimi-for-coding · updated 2026-07-14T04:31:40Z
Problem
Customer sessions could lose malformed records, unreadable files, partial uploads, tool arguments, identity, cost/token aliases, and agent-surface evidence.
The package also exposed a second hand-written recommendation/proposal model that duplicated the shared agent stack without being able to apply or compare exact changes.
Solution
@tangle-network/agent-eval@0.118.3for the telemetry vocabulary, execution accounting, deep analysts, evidence contracts, and confidence-bearing findings.Proof
pnpm typecheck: passed.pnpm test: 19 files, 198/198 tests passed.pnpm build: passed.pnpm check:package: packed CLI import/binary check passed.git diff --check: passed.Release
This removes public pre-1.0 SDK fields and must ship as
0.9.0.Merge with
[skip release], then dispatch the Publish workflow withrelease_type=minor; do not let the default main-push patch release mislabel the break.