refactor(agent): split runtime and harden loop

.github/workflows/prcheck.yml

@@ -83,6 +83,9 @@ jobs:
       - name: Check translations
         run: pnpm run i18n
 
+      - name: Native Agent behavior eval
+        run: pnpm run test:agent:eval
+
       - name: Build
         run: pnpm run build
 

docs/architecture/native-agent-evaluation/plan.md

@@ -0,0 +1,26 @@
+# Native Agent Evaluation Baseline — Plan
+
+## Harness
+
+Create a reusable scenario runner under `test/main/evals/` that constructs production `ProcessParams`,
+feeds scripted async provider rounds, records fake tool calls and message-store finalization, and
+returns a normalized report. Keep fixtures declarative so new regressions become scenarios rather
+than bespoke test setup.
+
+## Metrics and Results
+
+Each report includes scenario id, `ProcessResult` status/stop reason, provider-round count,
+tool-call count, elapsed milliseconds, usage, and final persisted status/metadata. Assertions remain
+explicit per scenario and distinguish exact expected calls from maximum budgets. Multi-round
+fixtures emit usage on every round so aggregation cannot pass by retaining only the final snapshot;
+a suite-level assertion verifies all registered scenarios ran and stayed within their budgets.
+
+## Integration
+
+Add `test:agent:eval` to `package.json`. The suite runs in the existing main-process Vitest project and
+reuses current Electron/event mocks. No generated report is committed.
+
+## Validation
+
+Run the dedicated eval command, targeted Agent runtime tests, node typecheck, formatting, i18n check,
+and lint.

docs/architecture/native-agent-evaluation/spec.md

@@ -0,0 +1,42 @@
+# Native Agent Evaluation Baseline
+
+## Problem
+
+The native Agent has broad unit coverage, but changes to the loop cannot currently be judged against
+one deterministic behavior baseline. Existing tests prove individual branches while providing no
+shared scenario contract or aggregate measures for task completion, tool use, pause, cancellation,
+and bounded-loop failure.
+
+## Goal
+
+Add an offline Vitest evaluation harness around the production `processStream` loop. It must use
+scripted provider events and fake tools, require no credentials or network, and fail CI when a stable
+behavior contract regresses.
+
+## Acceptance Criteria
+
+- A dedicated `test:agent:eval` command runs only native-Agent evaluation scenarios.
+- Scenarios cover direct completion, one and multiple tool rounds, tool failure, permission pause,
+  cancellation, pending-input yield, bounded provider/tool rounds, no-progress termination, empty
+  output, generic provider errors, and context errors.
+- Every scenario reports a normalized outcome and measures provider rounds, tool calls, elapsed time,
+  and token usage when supplied by the scripted provider.
+- Every scenario asserts the persisted run ID, outcome, stop reason, provider/tool counts, and all
+  supplied usage/cache-token fields rather than trusting harness-side counters alone.
+- Aggregate assertions cover success/outcome expectations and prevent extra provider or tool calls.
+- The harness never writes repository artifacts and is deterministic under fake timers.
+- Production Agent behavior is exercised through `processStream`; the harness does not duplicate the
+  loop implementation.
+
+## Constraints
+
+- No real provider calls, model snapshots, database migration, or renderer dependency.
+- Content quality and semantic task success remain future real-model evaluation work.
+- The harness validates the existing optional message-metadata accounting contract; it does not add
+  a renderer API or upload telemetry.
+
+## Non-Goals
+
+- Comparing model vendors or prompts using live credentials.
+- Adding a renderer dashboard or uploading traces.
+- Treating wall-clock performance from mocked tests as a production latency benchmark.

docs/architecture/native-agent-evaluation/tasks.md

@@ -0,0 +1,9 @@
+# Native Agent Evaluation Baseline — Tasks
+
+- [x] Inspect existing process-loop tests and memory eval conventions.
+- [x] Add reusable scripted provider/tool/message-store harness.
+- [x] Add deterministic behavior scenarios and aggregate budget assertions.
+- [x] Assert persisted run metadata, full usage/cache fields, and true multi-round aggregation.
+- [x] Separate exact expected calls from maximum call budgets.
+- [x] Add the `test:agent:eval` package command.
+- [ ] Run final unified repository quality gates.

docs/architecture/subagent-run-guardrails/plan.md

@@ -0,0 +1,37 @@
+# Subagent Run Guardrails - Plan
+
+## Runtime Changes
+
+- Extend the run schema with `runTimeoutMs`, default it at run creation, and publish the property in
+  the tool definition.
+- Store `runTimeoutMs`, `deadlineAt`, and optional `cancellationReason` on each run and include them
+  in serialized progress/final payloads.
+- Race task execution against a deadline promise. When the deadline fires, abort the run, mark and
+  resolve unfinished tasks, request cancellation for created child sessions, and resolve the run
+  lifecycle without waiting for stalled child setup or generation.
+- Clear the deadline timer when execution finishes first. Late child creation observes the aborted
+  controller and is cancelled before handoff.
+- Reject a fourth nonterminal run for the same parent before creating tasks or child sessions.
+
+## Handoff Contract
+
+- Always include a markdown template requiring the five standard result sections and `None` for
+  empty sections.
+- Append caller `expectedOutput` as an additional-requirements block so existing customization is
+  retained.
+
+## Compatibility
+
+- Keep all existing operations, task fields, progress/final payload keys, and tape finalization
+  decisions.
+- Continue treating `timeoutMs` solely as the polling timeout for `operation=wait`.
+- Do not add GitHub issue synchronization for this architecture slice.
+
+## Test Strategy
+
+- Use fake timers to prove a background run is cancelled and becomes waitable at its deadline.
+- Hold child cancellation pending to prove tape discard cannot race ahead of cancellation while the
+  deadline remains observable as terminal.
+- Verify three active runs are accepted, a fourth is rejected, and a terminal run frees capacity.
+- Assert timeout/deadline/cancellation fields in serialized status data and tool definition schema.
+- Assert handoffs contain all mandatory sections plus caller-provided additions.

docs/architecture/subagent-run-guardrails/spec.md

@@ -0,0 +1,46 @@
+# Subagent Run Guardrails - Spec
+
+> Status: **implemented; final validation pending**
+
+## Problem
+
+Native subagent runs currently have no lifetime limit. `timeoutMs` only limits how long an
+`operation=wait` call blocks, so background children can remain active indefinitely and a parent
+session can start an unbounded number of concurrent runs. Child handoffs also treat
+`expectedOutput` as a replacement for the default guidance, which makes aggregation quality
+inconsistent.
+
+## Requirements
+
+1. Every run has a deadline independent of `operation=wait`, configured by optional
+   `runTimeoutMs`. The default is 300000 ms (five minutes); accepted values are 1000-1800000 ms.
+2. At the deadline, all unfinished tasks become cancelled, created child sessions receive
+   cancellation, the run resolves even if child execution is still blocked, and the serialized run
+   records the cancellation reason.
+3. A parent session may have at most three nonterminal runs. Terminal runs retained for inspection
+   do not count toward the limit.
+4. Every child handoff requires `Result`, `Evidence`, `Changed Files`, `Validation`, and
+   `Unresolved` markdown sections. Empty sections use `None`.
+5. Caller-provided `expectedOutput` text is preserved as additional output guidance rather than
+   replacing the standard contract.
+
+## Acceptance Criteria
+
+- Foreground and background runs enforce the same deadline.
+- `list`, `info`, `log`, `wait`, and `kill` keep their existing meanings and run ownership checks.
+- Serialized run data includes the configured timeout, absolute deadline, and cancellation reason.
+- Manual cancellation, recursion prevention, and completed/error tape merge/discard behavior stay
+  compatible.
+- A cancelled child tape is not discarded until the child cancellation request settles; the run
+  deadline itself still resolves without waiting for a blocked cancellation request.
+- Focused fake-timer, capacity, serialization, and handoff tests pass.
+
+## Non-goals
+
+- Persisting run state across app restarts.
+- Changing subagent recursion rules, task count limits, or slot selection.
+- Adding per-child budgets or changing renderer IPC contracts.
+
+## Open Questions
+
+None.

docs/architecture/subagent-run-guardrails/tasks.md

@@ -0,0 +1,10 @@
+# Subagent Run Guardrails - Tasks
+
+- [x] T1: Define deadline, capacity, serialization, and handoff compatibility contracts.
+- [x] T2: Add validated run lifetime settings and serialized deadline metadata.
+- [x] T3: Enforce deadline cancellation and per-parent active-run capacity.
+- [x] T4: Strengthen the child output contract while preserving caller additions.
+- [x] T5: Add focused regression tests.
+- [ ] T6: Run final unified formatting, i18n validation, lint, typecheck, tests, and build.
+- [x] T7: Close late-child and handoff cancellation races while preserving tape ordering.
+- [x] T8: Make foreground parent cancellation observable without waiting for blocked child setup.

docs/features/mock-long-chat-debug-data/spec.md

@@ -11,7 +11,9 @@ Developers need a development-only control that creates a large realistic chat s
 - Clicking the button creates one regular chat session with 100 user messages and 100 assistant messages.
 - The created session uses the built-in `deepchat` agent, not a debug-only agent id.
 - Mock content is rewritten synthetic text, but can reuse the current database's message shapes and block types as samples.
-- Assistant messages include mixed short and long markdown plus supported block varieties such as content, reasoning, search, tool call, action, image, artifact thinking, plan, and error.
+- Assistant messages include mixed short and long markdown plus persisted block varieties such as
+  content, reasoning, search, tool call, action, image, artifact thinking, and error. Agent plans are
+  transient progress state and are not persisted into mock assistant history.
 - The inserted session appears in the session list without restarting the app.
 - The flow reports success or failure clearly to the developer.
 

docs/issues/agent-multiround-usage/spec.md

@@ -0,0 +1,59 @@
+# Agent Multi-Round Usage Accounting
+
+## Issue
+
+`processStream` can make multiple provider requests while an Agent executes tools. Every provider
+round emits its own usage event, but `accumulator.ts` currently overwrites message metadata with the
+latest event. The persisted message and usage dashboard therefore undercount multi-round Agent turns.
+
+## Impact
+
+- Token and cost statistics can represent only the last provider round.
+- Performance investigations cannot tell how many provider rounds or tool calls produced a message.
+- Deterministic Agent evaluations cannot compare orchestration efficiency using persisted metadata.
+
+## Root Cause
+
+Usage fields live only on `StreamState.metadata`. A usage event replaces those fields, and the loop
+does not maintain a per-round snapshot plus run aggregate.
+
+## Fix Plan
+
+- Track the latest usage event separately as `StreamState.roundUsage`.
+- At the end of each provider stream, add the round snapshot to run totals exactly once and clear it.
+- Persist opaque run identity, terminal outcome/stop reason, cumulative usage, provider-round count,
+  and executed-tool-call count in message metadata.
+- Commit an observed usage event before handling an exception so partial provider responses are not
+  lost; never infer usage when the provider emitted none.
+- Persist accounting when a turn pauses or aborts, then seed a resumed stream from the persisted
+  totals so one assistant message remains cumulative across user interactions.
+- Count a permission-approved deferred tool before resuming the provider stream.
+- Refuse a permission-approved deferred tool before invocation when its next count would exceed the
+  global 128-call budget.
+- Replace paused metadata with the actual terminal outcome when resume ends before `processStream`.
+
+## Compatibility
+
+No schema or IPC change is required because message metadata is stored as JSON. Existing consumers
+ignore the new optional fields. Single-round values remain unchanged.
+
+## Tasks
+
+- [x] Extend stream/message metadata types.
+- [x] Accumulate usage once per provider round.
+- [x] Add single-round, multi-round, missing-usage, and failure-path tests.
+- [x] Persist run identity and normalized terminal outcome metadata.
+- [x] Keep tool counts and terminal metadata correct across every interaction-resume exit.
+- [x] Keep deferred permission execution at or below the global tool-call cap.
+- [ ] Run final unified tests, typecheck, format, i18n, lint, and build.
+
+## Validation
+
+- Two provider rounds reporting 10 and 20 total tokens persist 30 total tokens.
+- Multiple usage events in one round use the latest cumulative event rather than summing snapshots.
+- Provider/tool counts match executed orchestration work.
+- Usage emitted before an `AbortError` remains persisted.
+- A permission pause persists its partial totals and the resumed run adds to them once.
+- A granted deferred tool increments the persisted cumulative tool count exactly once.
+- A deferred tool rejected at the global cap does not execute or increment the persisted count.
+- Resume cancellation and pre-stream failures no longer leave `runOutcome: paused` metadata.

docs/issues/agent-repeated-tool-loop/spec.md

@@ -0,0 +1,58 @@
+# Agent Repeated Tool Loop
+
+## Issue
+
+The native Agent can repeatedly request the same tool batch with equivalent arguments after the
+tools return exactly the same normalized results. The runtime currently allows that no-progress
+cycle to continue until another limit, such as the 128 tool-call cap, is reached.
+
+## Impact
+
+- Repeated calls consume provider tokens and tool execution time without advancing the task.
+- Side-effect-free tools can run many redundant times before the hard cap intervenes.
+- The provider receives no explicit signal that its previous strategy produced no new evidence.
+
+## Root Cause
+
+`processStream` tracks only the aggregate tool-call count. It does not compare consecutive completed
+tool batches, so semantically identical JSON arguments with different object-key order and identical
+normalized tool results are treated as ordinary progress.
+
+## Fix Plan
+
+- Fingerprint each fully executed batch from canonical tool names, stable JSON arguments, and a
+  compact hash of normalized tool-message contents. Normalize common generated IDs and timestamps
+  in results while preserving semantic arguments and payload fields.
+- Treat a changed call or result fingerprint as progress and reset the consecutive streak.
+- After the second identical batch, append one structured correction to the last tool message sent
+  to the next provider round, instructing the Agent to change strategy or finalize.
+- After the fourth consecutive identical batch, stop before another provider request, finalize an
+  error with `stopReason: 'no_progress'`, and terminally mark an open plan with `max_steps`.
+- Persist the compact streak snapshot and observe the completed paused batch on permission/question
+  resume so a pause cannot reset the guard.
+- Treat acknowledgement-only results such as `ok` as weak evidence: still issue the correction, but
+  do not hard-stop solely from those replies. The existing 128 tool-call cap remains the final bound.
+
+## Tasks
+
+- [x] Add the completed-batch fingerprint and consecutive-streak tracker.
+- [x] Integrate correction and termination behavior into `processStream`.
+- [x] Add deterministic process tests for correction, termination budgets, and streak resets.
+- [x] Preserve streaks across interaction resume and normalize volatile result fields.
+- [x] Avoid hard termination for acknowledgement-only result batches.
+- [x] Add a native-Agent evaluation scenario for the no-progress terminal path.
+- [ ] Run final unified tests, typecheck, format, i18n, lint, and build.
+
+## Validation
+
+- Stable-JSON-equivalent calls with the same result receive one correction after batch two.
+- Four consecutive identical call-and-result batches execute four provider rounds and four tool
+  calls, then return `no_progress` without a fifth provider call.
+- Changing either arguments or normalized results resets the streak.
+- Generated timestamps/IDs do not hide an otherwise repeated substantive result.
+- UUIDs in semantic payload fields remain progress rather than being erased as transport metadata.
+- Permission resume observes the completed paused batch without resetting the prior streak.
+- Constant acknowledgement-only results receive correction but do not cause a false hard stop.
+- The 128-call limit remains unchanged.
+
+No GitHub issue is linked or requested for this change.

docs/issues/agent-terminal-outcome-consistency/spec.md

@@ -0,0 +1,65 @@
+# Agent Terminal Outcome Consistency
+
+## Issue
+
+Several native Agent termination paths persist or return a terminal outcome that does not match the
+provider/runtime condition that ended the turn. A generic provider error event can be finalized as a
+successful message, while `max_tokens`, the tool-call hard cap, and context-window failures lose
+their specific stop reason or usage in `ProcessResult`.
+
+## Impact
+
+- Provider failures can appear as completed assistant messages and dispatch success-shaped hooks.
+- Persisted `runOutcome` / `runStopReason` can disagree with the visible terminal block.
+- Session-end hooks and deterministic evaluations lose the actual stop reason and cumulative usage.
+- Operational telemetry cannot distinguish a complete answer from a truncated or bounded run.
+
+## Root Cause
+
+`processStream` handles only context-window errors before its common success finalizer. Other error
+events fall through to `finalize()`. The common finalizer also stamps `complete` after loop exits
+without preserving why the loop stopped, and the context-window return omits fields already present
+in persisted metadata.
+
+## Fix Plan
+
+- Route every provider `error` stop through the error finalizer, retaining context-window handling.
+- Preserve explicit `max_tokens` and `max_tool_calls` reasons instead of overwriting them with
+  `complete`.
+- Return usage, stop reason, and error message for context-window failures.
+- Use the same canonical provider/tool/empty stop reason in persisted metadata, `ProcessResult`, and
+  terminal hooks.
+- Mark tool calls rejected by the 128-call budget as unexecuted errors before finalizing the message.
+- Reject permission-approved deferred execution before invocation when persisted accounting has
+  already reached the 128-call budget, without incrementing beyond the cap.
+- Give `processStream` sole ownership of terminal persistence after streaming starts; lifecycle code
+  owns only pre-stream cancellation/error settlement and includes run identity and zero-count
+  accounting there.
+- Add process-level and deterministic evaluation coverage for each terminal path.
+- Keep existing visible error-block and open-plan terminal behavior compatible.
+
+## Tasks
+
+- [x] Normalize generic provider error outcomes.
+- [x] Preserve max-token and tool-call-limit stop reasons.
+- [x] Return complete context-window failure metadata.
+- [x] Add focused process/evaluation regression scenarios.
+- [x] Align metadata, process results, and terminal hook stop reasons.
+- [x] Mark budget-skipped tool blocks and remove double stream-abort settlement.
+- [x] Enforce the same tool-call budget on deferred permission execution.
+- [x] Persist complete pre-stream error/abort run metadata.
+- [ ] Run final repository quality gates.
+
+## Validation
+
+- A generic provider error returns and persists `error`, never `completed`.
+- `max_tokens` and the 128-call guard retain distinct stop reasons.
+- Context-window errors return the same usage and reason persisted in message metadata.
+- Terminal hooks receive the normalized reason and usage.
+- A tool call that exceeds the hard cap is never finalized as a successful execution.
+- A granted deferred tool at the hard cap does not invoke the tool and keeps `toolCalls` at 128.
+- Stream abort writes one terminal message; pre-stream abort remains lifecycle-owned.
+
+## GitHub
+
+No GitHub issue sync was requested.