diff --git a/.github/workflows/prcheck.yml b/.github/workflows/prcheck.yml index 113a927f0..02358d19e 100644 --- a/.github/workflows/prcheck.yml +++ b/.github/workflows/prcheck.yml @@ -83,6 +83,9 @@ jobs: - name: Check translations run: pnpm run i18n + - name: Native Agent behavior eval + run: pnpm run test:agent:eval + - name: Build run: pnpm run build diff --git a/docs/architecture/native-agent-evaluation/plan.md b/docs/architecture/native-agent-evaluation/plan.md new file mode 100644 index 000000000..2b056d090 --- /dev/null +++ b/docs/architecture/native-agent-evaluation/plan.md @@ -0,0 +1,26 @@ +# Native Agent Evaluation Baseline — Plan + +## Harness + +Create a reusable scenario runner under `test/main/evals/` that constructs production `ProcessParams`, +feeds scripted async provider rounds, records fake tool calls and message-store finalization, and +returns a normalized report. Keep fixtures declarative so new regressions become scenarios rather +than bespoke test setup. + +## Metrics and Results + +Each report includes scenario id, `ProcessResult` status/stop reason, provider-round count, +tool-call count, elapsed milliseconds, usage, and final persisted status/metadata. Assertions remain +explicit per scenario and distinguish exact expected calls from maximum budgets. Multi-round +fixtures emit usage on every round so aggregation cannot pass by retaining only the final snapshot; +a suite-level assertion verifies all registered scenarios ran and stayed within their budgets. + +## Integration + +Add `test:agent:eval` to `package.json`. The suite runs in the existing main-process Vitest project and +reuses current Electron/event mocks. No generated report is committed. + +## Validation + +Run the dedicated eval command, targeted Agent runtime tests, node typecheck, formatting, i18n check, +and lint. diff --git a/docs/architecture/native-agent-evaluation/spec.md b/docs/architecture/native-agent-evaluation/spec.md new file mode 100644 index 000000000..dc9982828 --- /dev/null +++ b/docs/architecture/native-agent-evaluation/spec.md @@ -0,0 +1,42 @@ +# Native Agent Evaluation Baseline + +## Problem + +The native Agent has broad unit coverage, but changes to the loop cannot currently be judged against +one deterministic behavior baseline. Existing tests prove individual branches while providing no +shared scenario contract or aggregate measures for task completion, tool use, pause, cancellation, +and bounded-loop failure. + +## Goal + +Add an offline Vitest evaluation harness around the production `processStream` loop. It must use +scripted provider events and fake tools, require no credentials or network, and fail CI when a stable +behavior contract regresses. + +## Acceptance Criteria + +- A dedicated `test:agent:eval` command runs only native-Agent evaluation scenarios. +- Scenarios cover direct completion, one and multiple tool rounds, tool failure, permission pause, + cancellation, pending-input yield, bounded provider/tool rounds, no-progress termination, empty + output, generic provider errors, and context errors. +- Every scenario reports a normalized outcome and measures provider rounds, tool calls, elapsed time, + and token usage when supplied by the scripted provider. +- Every scenario asserts the persisted run ID, outcome, stop reason, provider/tool counts, and all + supplied usage/cache-token fields rather than trusting harness-side counters alone. +- Aggregate assertions cover success/outcome expectations and prevent extra provider or tool calls. +- The harness never writes repository artifacts and is deterministic under fake timers. +- Production Agent behavior is exercised through `processStream`; the harness does not duplicate the + loop implementation. + +## Constraints + +- No real provider calls, model snapshots, database migration, or renderer dependency. +- Content quality and semantic task success remain future real-model evaluation work. +- The harness validates the existing optional message-metadata accounting contract; it does not add + a renderer API or upload telemetry. + +## Non-Goals + +- Comparing model vendors or prompts using live credentials. +- Adding a renderer dashboard or uploading traces. +- Treating wall-clock performance from mocked tests as a production latency benchmark. diff --git a/docs/architecture/native-agent-evaluation/tasks.md b/docs/architecture/native-agent-evaluation/tasks.md new file mode 100644 index 000000000..d85923c71 --- /dev/null +++ b/docs/architecture/native-agent-evaluation/tasks.md @@ -0,0 +1,9 @@ +# Native Agent Evaluation Baseline — Tasks + +- [x] Inspect existing process-loop tests and memory eval conventions. +- [x] Add reusable scripted provider/tool/message-store harness. +- [x] Add deterministic behavior scenarios and aggregate budget assertions. +- [x] Assert persisted run metadata, full usage/cache fields, and true multi-round aggregation. +- [x] Separate exact expected calls from maximum call budgets. +- [x] Add the `test:agent:eval` package command. +- [ ] Run final unified repository quality gates. diff --git a/docs/architecture/subagent-run-guardrails/plan.md b/docs/architecture/subagent-run-guardrails/plan.md new file mode 100644 index 000000000..7750c272b --- /dev/null +++ b/docs/architecture/subagent-run-guardrails/plan.md @@ -0,0 +1,37 @@ +# Subagent Run Guardrails - Plan + +## Runtime Changes + +- Extend the run schema with `runTimeoutMs`, default it at run creation, and publish the property in + the tool definition. +- Store `runTimeoutMs`, `deadlineAt`, and optional `cancellationReason` on each run and include them + in serialized progress/final payloads. +- Race task execution against a deadline promise. When the deadline fires, abort the run, mark and + resolve unfinished tasks, request cancellation for created child sessions, and resolve the run + lifecycle without waiting for stalled child setup or generation. +- Clear the deadline timer when execution finishes first. Late child creation observes the aborted + controller and is cancelled before handoff. +- Reject a fourth nonterminal run for the same parent before creating tasks or child sessions. + +## Handoff Contract + +- Always include a markdown template requiring the five standard result sections and `None` for + empty sections. +- Append caller `expectedOutput` as an additional-requirements block so existing customization is + retained. + +## Compatibility + +- Keep all existing operations, task fields, progress/final payload keys, and tape finalization + decisions. +- Continue treating `timeoutMs` solely as the polling timeout for `operation=wait`. +- Do not add GitHub issue synchronization for this architecture slice. + +## Test Strategy + +- Use fake timers to prove a background run is cancelled and becomes waitable at its deadline. +- Hold child cancellation pending to prove tape discard cannot race ahead of cancellation while the + deadline remains observable as terminal. +- Verify three active runs are accepted, a fourth is rejected, and a terminal run frees capacity. +- Assert timeout/deadline/cancellation fields in serialized status data and tool definition schema. +- Assert handoffs contain all mandatory sections plus caller-provided additions. diff --git a/docs/architecture/subagent-run-guardrails/spec.md b/docs/architecture/subagent-run-guardrails/spec.md new file mode 100644 index 000000000..1685425f4 --- /dev/null +++ b/docs/architecture/subagent-run-guardrails/spec.md @@ -0,0 +1,46 @@ +# Subagent Run Guardrails - Spec + +> Status: **implemented; final validation pending** + +## Problem + +Native subagent runs currently have no lifetime limit. `timeoutMs` only limits how long an +`operation=wait` call blocks, so background children can remain active indefinitely and a parent +session can start an unbounded number of concurrent runs. Child handoffs also treat +`expectedOutput` as a replacement for the default guidance, which makes aggregation quality +inconsistent. + +## Requirements + +1. Every run has a deadline independent of `operation=wait`, configured by optional + `runTimeoutMs`. The default is 300000 ms (five minutes); accepted values are 1000-1800000 ms. +2. At the deadline, all unfinished tasks become cancelled, created child sessions receive + cancellation, the run resolves even if child execution is still blocked, and the serialized run + records the cancellation reason. +3. A parent session may have at most three nonterminal runs. Terminal runs retained for inspection + do not count toward the limit. +4. Every child handoff requires `Result`, `Evidence`, `Changed Files`, `Validation`, and + `Unresolved` markdown sections. Empty sections use `None`. +5. Caller-provided `expectedOutput` text is preserved as additional output guidance rather than + replacing the standard contract. + +## Acceptance Criteria + +- Foreground and background runs enforce the same deadline. +- `list`, `info`, `log`, `wait`, and `kill` keep their existing meanings and run ownership checks. +- Serialized run data includes the configured timeout, absolute deadline, and cancellation reason. +- Manual cancellation, recursion prevention, and completed/error tape merge/discard behavior stay + compatible. +- A cancelled child tape is not discarded until the child cancellation request settles; the run + deadline itself still resolves without waiting for a blocked cancellation request. +- Focused fake-timer, capacity, serialization, and handoff tests pass. + +## Non-goals + +- Persisting run state across app restarts. +- Changing subagent recursion rules, task count limits, or slot selection. +- Adding per-child budgets or changing renderer IPC contracts. + +## Open Questions + +None. diff --git a/docs/architecture/subagent-run-guardrails/tasks.md b/docs/architecture/subagent-run-guardrails/tasks.md new file mode 100644 index 000000000..3b5218e75 --- /dev/null +++ b/docs/architecture/subagent-run-guardrails/tasks.md @@ -0,0 +1,10 @@ +# Subagent Run Guardrails - Tasks + +- [x] T1: Define deadline, capacity, serialization, and handoff compatibility contracts. +- [x] T2: Add validated run lifetime settings and serialized deadline metadata. +- [x] T3: Enforce deadline cancellation and per-parent active-run capacity. +- [x] T4: Strengthen the child output contract while preserving caller additions. +- [x] T5: Add focused regression tests. +- [ ] T6: Run final unified formatting, i18n validation, lint, typecheck, tests, and build. +- [x] T7: Close late-child and handoff cancellation races while preserving tape ordering. +- [x] T8: Make foreground parent cancellation observable without waiting for blocked child setup. diff --git a/docs/features/mock-long-chat-debug-data/spec.md b/docs/features/mock-long-chat-debug-data/spec.md index 627701aaf..9bb53617f 100644 --- a/docs/features/mock-long-chat-debug-data/spec.md +++ b/docs/features/mock-long-chat-debug-data/spec.md @@ -11,7 +11,9 @@ Developers need a development-only control that creates a large realistic chat s - Clicking the button creates one regular chat session with 100 user messages and 100 assistant messages. - The created session uses the built-in `deepchat` agent, not a debug-only agent id. - Mock content is rewritten synthetic text, but can reuse the current database's message shapes and block types as samples. -- Assistant messages include mixed short and long markdown plus supported block varieties such as content, reasoning, search, tool call, action, image, artifact thinking, plan, and error. +- Assistant messages include mixed short and long markdown plus persisted block varieties such as + content, reasoning, search, tool call, action, image, artifact thinking, and error. Agent plans are + transient progress state and are not persisted into mock assistant history. - The inserted session appears in the session list without restarting the app. - The flow reports success or failure clearly to the developer. diff --git a/docs/issues/agent-multiround-usage/spec.md b/docs/issues/agent-multiround-usage/spec.md new file mode 100644 index 000000000..f63362fc6 --- /dev/null +++ b/docs/issues/agent-multiround-usage/spec.md @@ -0,0 +1,59 @@ +# Agent Multi-Round Usage Accounting + +## Issue + +`processStream` can make multiple provider requests while an Agent executes tools. Every provider +round emits its own usage event, but `accumulator.ts` currently overwrites message metadata with the +latest event. The persisted message and usage dashboard therefore undercount multi-round Agent turns. + +## Impact + +- Token and cost statistics can represent only the last provider round. +- Performance investigations cannot tell how many provider rounds or tool calls produced a message. +- Deterministic Agent evaluations cannot compare orchestration efficiency using persisted metadata. + +## Root Cause + +Usage fields live only on `StreamState.metadata`. A usage event replaces those fields, and the loop +does not maintain a per-round snapshot plus run aggregate. + +## Fix Plan + +- Track the latest usage event separately as `StreamState.roundUsage`. +- At the end of each provider stream, add the round snapshot to run totals exactly once and clear it. +- Persist opaque run identity, terminal outcome/stop reason, cumulative usage, provider-round count, + and executed-tool-call count in message metadata. +- Commit an observed usage event before handling an exception so partial provider responses are not + lost; never infer usage when the provider emitted none. +- Persist accounting when a turn pauses or aborts, then seed a resumed stream from the persisted + totals so one assistant message remains cumulative across user interactions. +- Count a permission-approved deferred tool before resuming the provider stream. +- Refuse a permission-approved deferred tool before invocation when its next count would exceed the + global 128-call budget. +- Replace paused metadata with the actual terminal outcome when resume ends before `processStream`. + +## Compatibility + +No schema or IPC change is required because message metadata is stored as JSON. Existing consumers +ignore the new optional fields. Single-round values remain unchanged. + +## Tasks + +- [x] Extend stream/message metadata types. +- [x] Accumulate usage once per provider round. +- [x] Add single-round, multi-round, missing-usage, and failure-path tests. +- [x] Persist run identity and normalized terminal outcome metadata. +- [x] Keep tool counts and terminal metadata correct across every interaction-resume exit. +- [x] Keep deferred permission execution at or below the global tool-call cap. +- [ ] Run final unified tests, typecheck, format, i18n, lint, and build. + +## Validation + +- Two provider rounds reporting 10 and 20 total tokens persist 30 total tokens. +- Multiple usage events in one round use the latest cumulative event rather than summing snapshots. +- Provider/tool counts match executed orchestration work. +- Usage emitted before an `AbortError` remains persisted. +- A permission pause persists its partial totals and the resumed run adds to them once. +- A granted deferred tool increments the persisted cumulative tool count exactly once. +- A deferred tool rejected at the global cap does not execute or increment the persisted count. +- Resume cancellation and pre-stream failures no longer leave `runOutcome: paused` metadata. diff --git a/docs/issues/agent-repeated-tool-loop/spec.md b/docs/issues/agent-repeated-tool-loop/spec.md new file mode 100644 index 000000000..7d1ee289e --- /dev/null +++ b/docs/issues/agent-repeated-tool-loop/spec.md @@ -0,0 +1,58 @@ +# Agent Repeated Tool Loop + +## Issue + +The native Agent can repeatedly request the same tool batch with equivalent arguments after the +tools return exactly the same normalized results. The runtime currently allows that no-progress +cycle to continue until another limit, such as the 128 tool-call cap, is reached. + +## Impact + +- Repeated calls consume provider tokens and tool execution time without advancing the task. +- Side-effect-free tools can run many redundant times before the hard cap intervenes. +- The provider receives no explicit signal that its previous strategy produced no new evidence. + +## Root Cause + +`processStream` tracks only the aggregate tool-call count. It does not compare consecutive completed +tool batches, so semantically identical JSON arguments with different object-key order and identical +normalized tool results are treated as ordinary progress. + +## Fix Plan + +- Fingerprint each fully executed batch from canonical tool names, stable JSON arguments, and a + compact hash of normalized tool-message contents. Normalize common generated IDs and timestamps + in results while preserving semantic arguments and payload fields. +- Treat a changed call or result fingerprint as progress and reset the consecutive streak. +- After the second identical batch, append one structured correction to the last tool message sent + to the next provider round, instructing the Agent to change strategy or finalize. +- After the fourth consecutive identical batch, stop before another provider request, finalize an + error with `stopReason: 'no_progress'`, and terminally mark an open plan with `max_steps`. +- Persist the compact streak snapshot and observe the completed paused batch on permission/question + resume so a pause cannot reset the guard. +- Treat acknowledgement-only results such as `ok` as weak evidence: still issue the correction, but + do not hard-stop solely from those replies. The existing 128 tool-call cap remains the final bound. + +## Tasks + +- [x] Add the completed-batch fingerprint and consecutive-streak tracker. +- [x] Integrate correction and termination behavior into `processStream`. +- [x] Add deterministic process tests for correction, termination budgets, and streak resets. +- [x] Preserve streaks across interaction resume and normalize volatile result fields. +- [x] Avoid hard termination for acknowledgement-only result batches. +- [x] Add a native-Agent evaluation scenario for the no-progress terminal path. +- [ ] Run final unified tests, typecheck, format, i18n, lint, and build. + +## Validation + +- Stable-JSON-equivalent calls with the same result receive one correction after batch two. +- Four consecutive identical call-and-result batches execute four provider rounds and four tool + calls, then return `no_progress` without a fifth provider call. +- Changing either arguments or normalized results resets the streak. +- Generated timestamps/IDs do not hide an otherwise repeated substantive result. +- UUIDs in semantic payload fields remain progress rather than being erased as transport metadata. +- Permission resume observes the completed paused batch without resetting the prior streak. +- Constant acknowledgement-only results receive correction but do not cause a false hard stop. +- The 128-call limit remains unchanged. + +No GitHub issue is linked or requested for this change. diff --git a/docs/issues/agent-terminal-outcome-consistency/spec.md b/docs/issues/agent-terminal-outcome-consistency/spec.md new file mode 100644 index 000000000..0782b62c5 --- /dev/null +++ b/docs/issues/agent-terminal-outcome-consistency/spec.md @@ -0,0 +1,65 @@ +# Agent Terminal Outcome Consistency + +## Issue + +Several native Agent termination paths persist or return a terminal outcome that does not match the +provider/runtime condition that ended the turn. A generic provider error event can be finalized as a +successful message, while `max_tokens`, the tool-call hard cap, and context-window failures lose +their specific stop reason or usage in `ProcessResult`. + +## Impact + +- Provider failures can appear as completed assistant messages and dispatch success-shaped hooks. +- Persisted `runOutcome` / `runStopReason` can disagree with the visible terminal block. +- Session-end hooks and deterministic evaluations lose the actual stop reason and cumulative usage. +- Operational telemetry cannot distinguish a complete answer from a truncated or bounded run. + +## Root Cause + +`processStream` handles only context-window errors before its common success finalizer. Other error +events fall through to `finalize()`. The common finalizer also stamps `complete` after loop exits +without preserving why the loop stopped, and the context-window return omits fields already present +in persisted metadata. + +## Fix Plan + +- Route every provider `error` stop through the error finalizer, retaining context-window handling. +- Preserve explicit `max_tokens` and `max_tool_calls` reasons instead of overwriting them with + `complete`. +- Return usage, stop reason, and error message for context-window failures. +- Use the same canonical provider/tool/empty stop reason in persisted metadata, `ProcessResult`, and + terminal hooks. +- Mark tool calls rejected by the 128-call budget as unexecuted errors before finalizing the message. +- Reject permission-approved deferred execution before invocation when persisted accounting has + already reached the 128-call budget, without incrementing beyond the cap. +- Give `processStream` sole ownership of terminal persistence after streaming starts; lifecycle code + owns only pre-stream cancellation/error settlement and includes run identity and zero-count + accounting there. +- Add process-level and deterministic evaluation coverage for each terminal path. +- Keep existing visible error-block and open-plan terminal behavior compatible. + +## Tasks + +- [x] Normalize generic provider error outcomes. +- [x] Preserve max-token and tool-call-limit stop reasons. +- [x] Return complete context-window failure metadata. +- [x] Add focused process/evaluation regression scenarios. +- [x] Align metadata, process results, and terminal hook stop reasons. +- [x] Mark budget-skipped tool blocks and remove double stream-abort settlement. +- [x] Enforce the same tool-call budget on deferred permission execution. +- [x] Persist complete pre-stream error/abort run metadata. +- [ ] Run final repository quality gates. + +## Validation + +- A generic provider error returns and persists `error`, never `completed`. +- `max_tokens` and the 128-call guard retain distinct stop reasons. +- Context-window errors return the same usage and reason persisted in message metadata. +- Terminal hooks receive the normalized reason and usage. +- A tool call that exceeds the hard cap is never finalized as a successful execution. +- A granted deferred tool at the hard cap does not invoke the tool and keeps `toolCalls` at 128. +- Stream abort writes one terminal message; pre-stream abort remains lifecycle-owned. + +## GitHub + +No GitHub issue sync was requested. diff --git a/docs/issues/mcp-tool-cancellation/spec.md b/docs/issues/mcp-tool-cancellation/spec.md new file mode 100644 index 000000000..14e014037 --- /dev/null +++ b/docs/issues/mcp-tool-cancellation/spec.md @@ -0,0 +1,83 @@ +# MCP Tool Cancellation + +## Issue + +Stopping a DeepChat Agent turn does not cancel an in-flight MCP tool request. The runtime passes an +`AbortSignal` into `ToolPresenter.callTool`, but the MCP branch drops it before calling +`McpPresenter`. The MCP SDK request can therefore continue until the server responds or the SDK's +default request timeout expires. + +## Impact + +- Agent Stop can remain blocked behind an MCP tool call. +- A remote MCP operation can continue after its parent generation has been cancelled. +- The runtime cannot reliably prevent another provider round until the outstanding tool call + settles. + +## Root Cause + +- `ToolPresenter.callTool` forwards the signal to built-in agent tools but omits it when routing to + `McpPresenter`. +- The concrete `McpPresenter`, `ToolManager`, and `McpClient` call paths do not accept or forward a + tool-call abort signal. +- `McpClient` invokes the SDK `client.callTool` without request options, despite the SDK supporting + an `AbortSignal`. + +## Fix Plan + +- Forward the existing optional `AbortSignal` through ToolPresenter, McpPresenter, ToolManager, and + McpClient. +- Reject pre-aborted calls before connecting to or invoking the MCP SDK client. +- Pass only `{ signal }` to the SDK request so its existing default timeout remains unchanged. +- Check or race the signal around connection, tool-definition refresh, argument preparation, and + CUA helper calls so cancellation is not delayed before the final SDK request. +- Observe the underlying Promise before an abort race so a late rejection after cancellation cannot + become an unhandled rejection. +- Treat abort failures as cancellation: do not run MCP session recovery, invalidate tool caches, or + retry the operation. Preserve the current handling of all non-abort failures. +- Classify a tool failure as parent cancellation only when the run signal is aborted; an + `AbortError` name alone remains a tool-local failure. +- Once tool execution returns a result, commit that result and its Tape facts before settling a + concurrently requested parent cancellation. +- Add focused forwarding and MCP client cancellation tests. + +## Constraints + +- No renderer, IPC, route, database, or persisted schema changes. +- No automatic retries or execution idempotency policy changes. +- No new tool timeout setting; retain the MCP SDK default request timeout. + +## Tasks + +- [x] Document the issue and implementation constraints. +- [x] Forward `AbortSignal` from ToolPresenter to McpPresenter. +- [x] Forward `AbortSignal` through ToolManager to McpClient. +- [x] Pass the signal into the MCP SDK request and preserve abort semantics. +- [x] Propagate cancellation through MCP preflight and helper awaits. +- [x] Keep the public `IToolPresenter` option contract aligned with the implementation. +- [x] Preserve returned tool results across late parent cancellation. +- [x] Keep tool-local `AbortError` failures inside the tool batch while the run signal is active. +- [x] Add focused regression tests. +- [ ] Run final unified tests, formatting, i18n validation, lint, typecheck, and build. + +## Validation + +- [x] ToolPresenter forwards the caller's exact signal to the MCP branch. +- [x] ToolManager forwards the caller's exact signal to the selected MCP client. +- [x] A pre-aborted MCP client call never invokes the SDK. +- [x] Aborting an in-flight MCP client call rejects promptly. +- [x] Abort failures do not trigger session recovery or tool-cache invalidation. +- [x] Existing non-abort MCP failure behavior remains covered and unchanged. +- [x] A native Agent turn cancellation reaches an in-flight MCP call through the real ToolPresenter. +- [x] A tool result returned as cancellation arrives is persisted before the run settles aborted. +- [x] A tool-local `AbortError` becomes a tool failure when the parent run remains active. +- [x] Abort winning a helper race consumes a later source rejection without an + `unhandledRejection`. +- [x] Standalone media cancellation consumes iterator teardown failures without an + `unhandledRejection`. + +Final validation commands are recorded after the unified repository gate run. + +## GitHub + +No GitHub issue sync was requested. diff --git a/package.json b/package.json index 98a584920..69263974a 100644 --- a/package.json +++ b/package.json @@ -20,6 +20,8 @@ "test:memory": "pnpm run test:memory:scope && pnpm run test:memory:type && vitest --config vitest.config.memory.ts --run", "test:memory:eval": "vitest --config vitest.config.memory-eval.ts --run", "test:main:memory-perf": "vitest --config vitest.config.memory-perf.ts --run", + "test:agent:eval": "vitest run --config vitest.config.ts test/main/evals/nativeAgent/nativeAgentBehavior.eval.test.ts", + "test:main:native-sqlite": "cross-env DEEPCHAT_REQUIRE_NATIVE_SQLITE=1 vitest --config vitest.config.ts test/main/presenter/agentMemoryTable.test.ts test/main/presenter/agentRuntimePresenter/tapeService.test.ts test/main/presenter/memoryRetrieval.eval.test.ts", "test:renderer": "vitest --config vitest.config.renderer.ts test/renderer", "test:coverage": "vitest --coverage", "test:watch": "vitest --watch", diff --git a/scripts/agent-cleanup-guard.mjs b/scripts/agent-cleanup-guard.mjs index 435b00f44..078fdb6f2 100644 --- a/scripts/agent-cleanup-guard.mjs +++ b/scripts/agent-cleanup-guard.mjs @@ -90,8 +90,10 @@ function extractModuleSpecifiers(source) { const specifiers = new Set() const patterns = [ /\bimport\s+(?:type\s+)?[\s\S]*?\bfrom\s*['"]([^'"]+)['"]/g, + /\bimport\s*['"]([^'"]+)['"]/g, /\bexport\s+[\s\S]*?\bfrom\s*['"]([^'"]+)['"]/g, - /\bimport\s*\(\s*['"]([^'"]+)['"]\s*\)/g + /\bimport\s*\(\s*['"]([^'"]+)['"]\s*\)/g, + /\brequire\s*\(\s*['"]([^'"]+)['"]\s*\)/g ] for (const pattern of patterns) { diff --git a/src/main/agent/deepchat/instance/deepChatAgentInstance.ts b/src/main/agent/deepchat/instance/deepChatAgentInstance.ts index 049e39fa6..e435bc2b7 100644 --- a/src/main/agent/deepchat/instance/deepChatAgentInstance.ts +++ b/src/main/agent/deepchat/instance/deepChatAgentInstance.ts @@ -179,6 +179,10 @@ export class DeepChatAgentInstance { } registerActiveGeneration(run: LoopRun): LoopRun { + const previousController = this.activeRun?.abortController ?? this.abortController + if (previousController && previousController !== run.abortController) { + previousController.abort() + } this.activeRun = run this.abortController = run.abortController return run diff --git a/src/main/agent/deepchat/loop/contextCoordinator.ts b/src/main/agent/deepchat/loop/contextCoordinator.ts index 0aacc1a17..27faecf10 100644 --- a/src/main/agent/deepchat/loop/contextCoordinator.ts +++ b/src/main/agent/deepchat/loop/contextCoordinator.ts @@ -135,6 +135,7 @@ export interface ProviderAttemptStreamPort { maxTokens: number tools: MCPToolDefinition[] }): AsyncGenerator + assertAvailable?(): void beforeStream(): void } @@ -377,6 +378,7 @@ export class DeepChatContextCoordinator { try { providerAttemptLoop: for (;;) { + input.provider.assertAvailable?.() const strictProviderOverflowRetry = strictProviderOverflowRetryPending strictProviderOverflowRetryPending = false const { providerMessages, providerMaxTokens } = await prepareProviderAttempt({ @@ -415,11 +417,13 @@ export class DeepChatContextCoordinator { preflightContextRecoveryAttempted || input.run.providerRecovery.contextOverflowHandoffAttempted ) { + input.provider.assertAvailable?.() if (!scheduleStrictProviderOverflowRetry()) { throw buildProviderOverflowRetryFailure(providerMessages, providerMaxTokens) } continue providerAttemptLoop } + input.provider.assertAvailable?.() await recoverProviderContextOverflow(providerMessages, providerMaxTokens) continue providerAttemptLoop } @@ -443,11 +447,13 @@ export class DeepChatContextCoordinator { preflightContextRecoveryAttempted || input.run.providerRecovery.contextOverflowHandoffAttempted ) { + input.provider.assertAvailable?.() if (!scheduleStrictProviderOverflowRetry()) { throw buildProviderOverflowRetryFailure(providerMessages, providerMaxTokens) } continue providerAttemptLoop } + input.provider.assertAvailable?.() await recoverProviderContextOverflow(providerMessages, providerMaxTokens) continue providerAttemptLoop } diff --git a/src/main/agent/deepchat/loop/deepChatLoopEngine.ts b/src/main/agent/deepchat/loop/deepChatLoopEngine.ts index a801d867a..1a3557949 100644 --- a/src/main/agent/deepchat/loop/deepChatLoopEngine.ts +++ b/src/main/agent/deepchat/loop/deepChatLoopEngine.ts @@ -1,6 +1,6 @@ import { enterProviderRound, type LoopRun } from './loopRun' -const MAX_TOOL_CALLS = 128 +export const MAX_TOOL_CALLS = 128 export type ProviderRoundOutcome = | { type: 'terminal' } @@ -20,6 +20,7 @@ export type DeepChatLoopOutcome = export interface DeepChatLoopDependencies { maxProviderRounds?: number + initialExecutedToolCount?: number consumeProviderRound(input: { run: LoopRun providerRound: number @@ -74,7 +75,11 @@ export class DeepChatLoopEngine { Number.isInteger(dependencies.maxProviderRounds) && dependencies.maxProviderRounds! > 0 ? dependencies.maxProviderRounds! : Number.POSITIVE_INFINITY - let executedToolCount = 0 + let executedToolCount = + Number.isInteger(dependencies.initialExecutedToolCount) && + dependencies.initialExecutedToolCount! > 0 + ? dependencies.initialExecutedToolCount! + : 0 while (true) { const providerRound = enterProviderRound(run) diff --git a/src/main/agent/deepchat/loop/ports.ts b/src/main/agent/deepchat/loop/ports.ts index 216ca2268..4f7324f94 100644 --- a/src/main/agent/deepchat/loop/ports.ts +++ b/src/main/agent/deepchat/loop/ports.ts @@ -39,7 +39,7 @@ export type ToolExecutionOptions = ToolCallOptions export interface ToolExecutionPort { preCheck?( call: MCPToolCall, - options?: Pick + options?: Pick ): Promise execute( call: MCPToolCall, diff --git a/src/main/agent/deepchat/memory/memoryRuntimeCoordinator.ts b/src/main/agent/deepchat/memory/memoryRuntimeCoordinator.ts index 6cc94f801..b7ebe872c 100644 --- a/src/main/agent/deepchat/memory/memoryRuntimeCoordinator.ts +++ b/src/main/agent/deepchat/memory/memoryRuntimeCoordinator.ts @@ -733,6 +733,7 @@ export class MemoryRuntimeCoordinator implements MemoryPromptContributor, Memory private extractPlainText(record: Pick): string { try { const parsed = JSON.parse(record.content) as unknown + if (typeof parsed === 'string') return parsed.trim() if (record.role === 'user') { const text = (parsed as { text?: unknown })?.text return typeof text === 'string' ? text.trim() : '' @@ -751,7 +752,7 @@ export class MemoryRuntimeCoordinator implements MemoryPromptContributor, Memory } return '' } catch { - return '' + return record.content.trim() } } } diff --git a/src/main/lib/awaitWithAbort.ts b/src/main/lib/awaitWithAbort.ts new file mode 100644 index 000000000..1cb796ec8 --- /dev/null +++ b/src/main/lib/awaitWithAbort.ts @@ -0,0 +1,30 @@ +export async function awaitWithAbort(promise: Promise, signal?: AbortSignal): Promise { + if (!signal) { + return await promise + } + + // Observe the source before checking the signal. The caller may have created the promise in an + // expression that synchronously aborted the signal, and that source can still reject later. + void promise.catch(() => undefined) + signal.throwIfAborted() + + let abortListener: (() => void) | undefined + try { + return await Promise.race([ + promise, + new Promise((_, reject) => { + abortListener = () => { + try { + signal.throwIfAborted() + } catch (error) { + reject(error) + } + } + signal.addEventListener('abort', abortListener, { once: true }) + if (signal.aborted) abortListener() + }) + ]) + } finally { + if (abortListener) signal.removeEventListener('abort', abortListener) + } +} diff --git a/src/main/lib/toolCallImagePreviews.ts b/src/main/lib/toolCallImagePreviews.ts index b4ae132e2..a34cbc834 100644 --- a/src/main/lib/toolCallImagePreviews.ts +++ b/src/main/lib/toolCallImagePreviews.ts @@ -1,4 +1,5 @@ import type { MCPContentItem, ToolCallImagePreview } from '@shared/types/core/mcp' +import { awaitWithAbort } from './awaitWithAbort' type ImagePreviewInput = { data: string @@ -12,6 +13,7 @@ type ExtractToolCallImagePreviewsParams = { toolArgs?: string content: string | MCPContentItem[] cacheImage?: (data: string) => Promise + signal?: AbortSignal } const DATA_IMAGE_URL_PATTERN = /data:image\/[a-zA-Z0-9.+-]+;base64,[a-zA-Z0-9+/=\r\n]+/g @@ -88,17 +90,20 @@ function normalizeImagePayload(data: string, mimeType: string): string { async function cachePreviewData( data: string, - cacheImage?: (data: string) => Promise + cacheImage?: (data: string) => Promise, + signal?: AbortSignal ): Promise { if (!cacheImage) { return undefined } try { - const cachedData = await cacheImage(data) + signal?.throwIfAborted() + const cachedData = await awaitWithAbort(cacheImage(data), signal) const cachedDataTrimmed = cachedData.trim().toLowerCase() return cachedDataTrimmed.startsWith('data:image/') ? undefined : cachedData - } catch { + } catch (error) { + if (signal?.aborted) throw error return undefined } } @@ -216,6 +221,7 @@ function extractStructuredImagePreviews(content: MCPContentItem[]): ImagePreview export async function extractToolCallImagePreviews( params: ExtractToolCallImagePreviewsParams ): Promise { + params.signal?.throwIfAborted() const inputs: ImagePreviewInput[] = [] const screenshotPreview = extractScreenshotPreview( params.toolName, @@ -235,7 +241,8 @@ export async function extractToolCallImagePreviews( const previews: ToolCallImagePreview[] = [] const seen = new Set() for (const input of inputs) { - const data = await cachePreviewData(input.data, params.cacheImage) + params.signal?.throwIfAborted() + const data = await cachePreviewData(input.data, params.cacheImage, params.signal) if (data && seen.has(data)) { continue } @@ -251,5 +258,6 @@ export async function extractToolCallImagePreviews( }) } + params.signal?.throwIfAborted() return previews } diff --git a/src/main/presenter/agentRuntimePresenter/accumulator.ts b/src/main/presenter/agentRuntimePresenter/accumulator.ts index 5a1a8d2d4..b2a9cf90f 100644 --- a/src/main/presenter/agentRuntimePresenter/accumulator.ts +++ b/src/main/presenter/agentRuntimePresenter/accumulator.ts @@ -3,6 +3,26 @@ import type { LLMCoreStreamEvent } from '@shared/types/core/llm-events' import type { ChatMessageProviderOptions } from '@shared/types/core/chat-message' import type { StreamState } from './types' +export function commitRoundUsage(state: StreamState): void { + const roundUsage = state.roundUsage + if (!roundUsage) { + return + } + + state.metadata.inputTokens = (state.metadata.inputTokens ?? 0) + roundUsage.inputTokens + state.metadata.outputTokens = (state.metadata.outputTokens ?? 0) + roundUsage.outputTokens + state.metadata.totalTokens = (state.metadata.totalTokens ?? 0) + roundUsage.totalTokens + if (typeof roundUsage.cachedInputTokens === 'number') { + state.metadata.cachedInputTokens = + (state.metadata.cachedInputTokens ?? 0) + roundUsage.cachedInputTokens + } + if (typeof roundUsage.cacheWriteInputTokens === 'number') { + state.metadata.cacheWriteInputTokens = + (state.metadata.cacheWriteInputTokens ?? 0) + roundUsage.cacheWriteInputTokens + } + state.roundUsage = null +} + export function finalizeTrailingPendingNarrativeBlocks(blocks: AssistantMessageBlock[]): boolean { const last = blocks[blocks.length - 1] if ( @@ -221,15 +241,23 @@ export function accumulate(state: StreamState, event: LLMCoreStreamEvent): void break } case 'usage': { - state.metadata.inputTokens = event.usage.prompt_tokens - state.metadata.outputTokens = event.usage.completion_tokens - state.metadata.totalTokens = event.usage.total_tokens - state.metadata.cachedInputTokens = event.usage.cached_tokens - state.metadata.cacheWriteInputTokens = event.usage.cache_write_tokens + // Providers may emit more than one cumulative usage snapshot for a request. Keep only the + // latest snapshot for this provider round; processStream commits it once before the next round. + state.roundUsage = { + inputTokens: event.usage.prompt_tokens, + outputTokens: event.usage.completion_tokens, + totalTokens: event.usage.total_tokens, + cachedInputTokens: event.usage.cached_tokens, + cacheWriteInputTokens: event.usage.cache_write_tokens + } break } case 'stop': { - state.stopReason = mapStopReason(event.stop_reason) + // Keep an explicit stream error terminal reason even if the provider later emits a + // generic complete stop after the error event. + if (state.stopReason !== 'error') { + state.stopReason = mapStopReason(event.stop_reason) + } break } case 'error': { diff --git a/src/main/presenter/agentRuntimePresenter/compactionService.ts b/src/main/presenter/agentRuntimePresenter/compactionService.ts index 02dbfadc8..31731f6de 100644 --- a/src/main/presenter/agentRuntimePresenter/compactionService.ts +++ b/src/main/presenter/agentRuntimePresenter/compactionService.ts @@ -9,6 +9,7 @@ import type { import type { ChatMessage } from '@shared/types/core/chat-message' import type { IConfigPresenter, ILlmProviderPresenter } from '@shared/presenter' import type { DeepChatMessageStore } from './messageStore' +import { awaitWithAbort } from '@/lib/awaitWithAbort' import type { DeepChatSessionStore, ReconstructionAnchorPromptState, @@ -335,7 +336,10 @@ export class CompactionService { signal?: AbortSignal }): Promise { throwIfAbortRequested(params.signal) - const settings = await this.getCompactionSettings(params.sessionId) + const settings = await awaitWithAbort( + this.getCompactionSettings(params.sessionId), + params.signal + ) throwIfAbortRequested(params.signal) if (!settings.enabled) { return null @@ -380,7 +384,10 @@ export class CompactionService { signal?: AbortSignal }): Promise { throwIfAbortRequested(params.signal) - const settings = await this.getCompactionSettings(params.sessionId) + const settings = await awaitWithAbort( + this.getCompactionSettings(params.sessionId), + params.signal + ) throwIfAbortRequested(params.signal) if (!settings.enabled) { return null @@ -431,7 +438,10 @@ export class CompactionService { signal?: AbortSignal }): Promise { throwIfAbortRequested(params.signal) - const settings = await this.getCompactionSettings(params.sessionId) + const settings = await awaitWithAbort( + this.getCompactionSettings(params.sessionId), + params.signal + ) throwIfAbortRequested(params.signal) if (!settings.enabled) { return null @@ -502,6 +512,7 @@ export class CompactionService { reserveTokens: intent.reserveTokens, signal }) + throwIfAbortRequested(signal) const summaryUpdatedAt = Date.now() const updatedState: SessionSummaryState = { @@ -758,7 +769,10 @@ export class CompactionService { }): Promise { throwIfAbortRequested(params.signal) const currentModel = params.currentModel - const assistantModel = await this.getAssistantModelSpec(params.sessionId, currentModel) + const assistantModel = await awaitWithAbort( + this.getAssistantModelSpec(params.sessionId, currentModel), + params.signal + ) throwIfAbortRequested(params.signal) const previousSummaryTokens = approximateTokenSize(params.previousSummary || '') const blockTokens = params.summaryBlocks.reduce( @@ -982,13 +996,17 @@ export class CompactionService { await this.llmProviderPresenter.executeWithRateLimit(model.providerId) } throwIfAbortRequested(signal) - const response = await this.llmProviderPresenter.generateText( - model.providerId, - prompt, - model.modelId, - 0.2, - this.getSummaryOutputTokens(reserveTokens) + const response = await awaitWithAbort( + this.llmProviderPresenter.generateText( + model.providerId, + prompt, + model.modelId, + 0.2, + this.getSummaryOutputTokens(reserveTokens) + ), + signal ) + throwIfAbortRequested(signal) const summary = sanitizeSummaryContent(response.content || '') if (!summary) { throw new Error('Compaction summary generation returned empty content.') diff --git a/src/main/presenter/agentRuntimePresenter/dispatch.ts b/src/main/presenter/agentRuntimePresenter/dispatch.ts index 6cd77b623..1e3f359c3 100644 --- a/src/main/presenter/agentRuntimePresenter/dispatch.ts +++ b/src/main/presenter/agentRuntimePresenter/dispatch.ts @@ -642,6 +642,37 @@ function buildToolErrorOutcome( } } +function buildReturnedToolResultOutcome( + execution: ToolExecutionContext, + rawData: MCPToolResponse +): Extract { + const responseText = toolResponseToText(rawData.content) + const isError = rawData.isError === true + return { + kind: 'staged', + stagedResult: { + toolCallId: execution.completedToolCall.id, + toolName: execution.completedToolCall.name, + toolSource: execution.toolDef?.source, + serverName: execution.toolContext.serverName, + toolArgs: execution.completedToolCall.arguments, + responseText, + isError, + searchPayload: extractSearchPayload( + rawData.content, + execution.toolContext.name, + execution.toolContext.serverName + ), + rtkApplied: rawData.rtkApplied, + rtkMode: rawData.rtkMode, + rtkFallbackReason: rawData.rtkFallbackReason, + imagePreviews: rawData.imagePreviews, + postHookKind: isError ? 'failure' : 'success' + }, + toolsChanged: false + } +} + function scheduleRendererFlush( state: StreamState, rendererFlushHandle?: Pick @@ -1028,6 +1059,7 @@ async function reviewAutoApproveAction(params: { permission, reason }) + io.abortSignal.throwIfAborted() if (!result || result.decision === 'ask_user') { return 'ask_user' @@ -1248,6 +1280,7 @@ async function runToolCall(params: { state: StreamState rendererFlushHandle: RendererFlushHandle allowProgressUpdates: boolean + onToolCallStarted?: (toolCallId: string) => void }): Promise { const { execution, @@ -1259,11 +1292,14 @@ async function runToolCall(params: { io, state, rendererFlushHandle, - allowProgressUpdates + allowProgressUpdates, + onToolCallStarted } = params const { completedToolCall, toolCall, toolContext } = execution + let returnedToolResult: MCPToolResponse | null = null try { + io.abortSignal.throwIfAborted() const applyProgressUpdate = (update: AgentToolProgressUpdate) => { if ( update.kind === 'agent_plan' && @@ -1302,19 +1338,28 @@ async function runToolCall(params: { scheduleRendererFlush(state, rendererFlushHandle) } - const callTool = async () => - await toolExecution.execute(toolCall, { + let toolCallStarted = false + const callTool = async () => { + io.abortSignal.throwIfAborted() + if (!toolCallStarted) { + toolCallStarted = true + onToolCallStarted?.(completedToolCall.id) + } + const result = await toolExecution.execute(toolCall, { onProgress: applyProgressUpdate, signal: io.abortSignal, permissionMode: toolPermissionMode, activeSkillNames: controls?.getActiveSkillNames?.(), enabledSkillNames: controls?.getEnabledSkillNames?.() }) + return result + } let toolCallResult = await callTool() let toolRawData = toolCallResult.rawData if (toolRawData?.requiresPermission) { + io.abortSignal.throwIfAborted() const pendingPermission = normalizePermissionRequest( toolRawData.permissionRequest as PermissionRequestLike | undefined, { @@ -1360,6 +1405,11 @@ async function runToolCall(params: { } } + returnedToolResult = toolRawData + if (io.abortSignal.aborted) { + return buildReturnedToolResultOutcome(execution, toolRawData) + } + const subagentState = extractSubagentToolState(toolRawData) if (allowProgressUpdates && (subagentState.subagentProgress || subagentState.subagentFinal)) { updateSubagentToolCallBlock( @@ -1379,7 +1429,8 @@ async function runToolCall(params: { toolName: completedToolCall.name, toolArgs: completedToolCall.arguments, content: toolRawData.content, - cacheImage: controls?.cacheImage + cacheImage: controls?.cacheImage, + signal: io.abortSignal })) toolRawData = { @@ -1394,6 +1445,7 @@ async function runToolCall(params: { signal: io.abortSignal }) } + io.abortSignal.throwIfAborted() const searchPayload = extractSearchPayload( toolRawData.content, @@ -1408,6 +1460,7 @@ async function runToolCall(params: { toolName: toolContext.name, rawContent: responseText }) + io.abortSignal.throwIfAborted() const stagedResponseText = preparedResult.kind === 'tool_error' ? preparedResult.message : preparedResult.content const stagedIsError = preparedResult.kind === 'tool_error' || toolRawData.isError === true @@ -1415,6 +1468,7 @@ async function runToolCall(params: { const activatedSkill = extractActivatedSkillAfterCall(completedToolCall.name, toolRawData) if (activatedSkill) { await controls?.activateSkill?.(activatedSkill) + io.abortSignal.throwIfAborted() } return { @@ -1439,6 +1493,10 @@ async function runToolCall(params: { toolsChanged: Boolean(activatedSkill) } } catch (err) { + if (io.abortSignal.aborted && returnedToolResult) { + return buildReturnedToolResultOutcome(execution, returnedToolResult) + } + if (io.abortSignal.aborted) throw err return buildToolErrorOutcome(execution, err) } } @@ -1460,7 +1518,8 @@ export async function executeTools( collaborators?: ToolDispatchCollaborators, providerId?: string ): Promise> { - const { notificationObserver, controls, diagnostics } = collaborators ?? {} + const { notificationObserver, controls, diagnostics, onToolCallStarted } = collaborators ?? {} + io.abortSignal.throwIfAborted() finalizePendingNarrativeBeforeToolExecution(state) persistToolExecutionState(io, state, rendererFlushHandle) const toolPermissionMode = getToolCapabilityPermissionMode(permissionMode) @@ -1538,13 +1597,15 @@ export async function executeTools( buildToolExecutionContext(tc, tools, io.sessionId, providerId) ) - const outcomes = await Promise.all( + const settledOutcomes = await Promise.allSettled( executions.map(async (execution) => { try { if (toolExecution.preCheck) { const preChecked = await toolExecution.preCheck(execution.toolCall, { - permissionMode: toolPermissionMode + permissionMode: toolPermissionMode, + signal: io.abortSignal }) + io.abortSignal.throwIfAborted() if (preChecked?.needsPermission) { const permission = normalizePermissionRequest(preChecked as PermissionRequestLike, { toolName: execution.toolContext.name, @@ -1553,6 +1614,7 @@ export async function executeTools( }) if (permission) { await autoGrantPermission(controls, io.sessionId, permission) + io.abortSignal.throwIfAborted() } } } @@ -1576,13 +1638,26 @@ export async function executeTools( io, state, rendererFlushHandle, - allowProgressUpdates: false + allowProgressUpdates: false, + onToolCallStarted }) } catch (error) { + if (io.abortSignal.aborted) throw error return buildToolErrorOutcome(execution, error) } }) ) + const outcomes: ToolRunOutcome[] = [] + let cancellationError: unknown + for (const outcome of settledOutcomes) { + if (outcome.status === 'fulfilled') { + outcomes.push(outcome.value) + } else if (io.abortSignal.aborted) { + cancellationError ??= outcome.reason + } else { + throw outcome.reason + } + } for (const outcome of outcomes) { batchState.invokedCallIds.add( @@ -1617,6 +1692,10 @@ export async function executeTools( executed += 1 } + if (cancellationError && stagedResults.length === 0) { + throw cancellationError + } + if (stagedResults.length > 0) { const fittedResults = await toolResults.fitBatch({ conversationMessages: conversation, @@ -1631,7 +1710,6 @@ export async function executeTools( contextLength, maxTokens }) - const finalizedInteractions = applyFinalizedToolResults({ stagedResults, fittedResults: fittedResults.results, @@ -1664,7 +1742,10 @@ export async function executeTools( } for (const tc of state.completedToolCalls) { - if (io.abortSignal.aborted) break + if (io.abortSignal.aborted && stagedResults.length > 0) { + break + } + io.abortSignal.throwIfAborted() const execution = buildToolExecutionContext(tc, tools, io.sessionId, providerId) const { toolCall, toolContext } = execution @@ -1710,8 +1791,10 @@ export async function executeTools( let preCheckedPermission: PendingToolInteraction['permission'] | null = null if (toolExecution.preCheck) { const preChecked = await toolExecution.preCheck(toolCall, { - permissionMode: toolPermissionMode + permissionMode: toolPermissionMode, + signal: io.abortSignal }) + io.abortSignal.throwIfAborted() if (preChecked?.needsPermission) { preCheckedPermission = normalizePermissionRequest(preChecked as PermissionRequestLike, { toolName: toolContext.name, @@ -1724,6 +1807,7 @@ export async function executeTools( if (preCheckedPermission) { if (permissionMode === 'full_access') { await autoGrantPermission(controls, io.sessionId, preCheckedPermission) + io.abortSignal.throwIfAborted() } else if (permissionMode === 'auto_approve') { const review = await reviewAutoApproveAction({ controls, @@ -1736,6 +1820,7 @@ export async function executeTools( }) if (review === 'auto_allow') { await autoGrantPermission(controls, io.sessionId, preCheckedPermission) + io.abortSignal.throwIfAborted() } else { emitDeepChatLoopNotification(notificationObserver, { event: 'PermissionRequest', @@ -1843,7 +1928,8 @@ export async function executeTools( io, state, rendererFlushHandle, - allowProgressUpdates: true + allowProgressUpdates: true, + onToolCallStarted }) batchState.invokedCallIds.add(tc.id) @@ -1875,6 +1961,12 @@ export async function executeTools( toolsChanged = toolsChanged || outcome.toolsChanged executed += 1 } catch (err) { + if (io.abortSignal.aborted) { + if (stagedResults.length > 0) { + break + } + throw err + } const errorText = err instanceof Error ? err.message : String(err) stagedResults.push({ toolCallId: tc.id, @@ -1903,7 +1995,6 @@ export async function executeTools( contextLength, maxTokens }) - const finalizedInteractions = applyFinalizedToolResults({ stagedResults, fittedResults: fittedResults.results, @@ -1935,6 +2026,18 @@ export async function executeTools( return buildToolBatchOutcome(batchState, pendingInteractions, executed, toolsChanged) } +function stampGenerationTiming(state: StreamState): void { + state.metadata.generationTime = Math.max(0, Date.now() - state.startTime) + if (state.firstTokenTime !== null) { + state.metadata.firstTokenTime = Math.max(0, state.firstTokenTime - state.startTime) + } + if (state.metadata.outputTokens && state.metadata.generationTime > 0) { + state.metadata.tokensPerSecond = Math.round( + (state.metadata.outputTokens / state.metadata.generationTime) * 1000 + ) + } +} + export function finalizePaused(state: StreamState, io: IoParams): void { for (const block of state.blocks) { if ( @@ -1949,7 +2052,9 @@ export function finalizePaused(state: StreamState, io: IoParams): void { } } - io.messageStore.updateAssistantContent(io.messageId, state.blocks) + stampGenerationTiming(state) + + io.messageStore.updateAssistantContent(io.messageId, state.blocks, JSON.stringify(state.metadata)) flushBlocksToRenderer(io, state.blocks) publishDeepchatEvent('chat.stream.completed', { requestId: io.requestId, @@ -1965,16 +2070,7 @@ export function finalize(state: StreamState, io: IoParams): void { } stampPlanTerminalIfOpen(state, io, state.planTerminalReason) - const endTime = Date.now() - state.metadata.generationTime = endTime - state.startTime - if (state.firstTokenTime !== null) { - state.metadata.firstTokenTime = state.firstTokenTime - state.startTime - } - if (state.metadata.outputTokens && state.metadata.generationTime > 0) { - state.metadata.tokensPerSecond = Math.round( - (state.metadata.outputTokens / state.metadata.generationTime) * 1000 - ) - } + stampGenerationTiming(state) io.messageStore.finalizeAssistantMessage( io.messageId, @@ -1999,16 +2095,7 @@ export function finalizeError(state: StreamState, io: IoParams, error: unknown): errorMessage === USER_CANCELED_GENERATION_ERROR ? 'aborted' : 'error' ) - const endTime = Date.now() - state.metadata.generationTime = endTime - state.startTime - if (state.firstTokenTime !== null) { - state.metadata.firstTokenTime = state.firstTokenTime - state.startTime - } - if (state.metadata.outputTokens && state.metadata.generationTime > 0) { - state.metadata.tokensPerSecond = Math.round( - (state.metadata.outputTokens / state.metadata.generationTime) * 1000 - ) - } + stampGenerationTiming(state) io.messageStore.setMessageError(io.messageId, state.blocks, JSON.stringify(state.metadata)) flushBlocksToRenderer(io, state.blocks) diff --git a/src/main/presenter/agentRuntimePresenter/index.ts b/src/main/presenter/agentRuntimePresenter/index.ts index 4ddf52475..ce9047871 100644 --- a/src/main/presenter/agentRuntimePresenter/index.ts +++ b/src/main/presenter/agentRuntimePresenter/index.ts @@ -14,6 +14,7 @@ import type { ChatMessagePageResult, ChatMessageRecord, DeepChatSessionState, + MessageMetadata, MessagePageCursor, MessageStartResult, MessageFile, @@ -88,6 +89,7 @@ import { buildSystemEnvPrompt } from '@/agent/deepchat/resources/systemEnvPromptBuilder' import { createLoopRun, type LoopRun } from '@/agent/deepchat/loop/loopRun' +import { MAX_TOOL_CALLS } from '@/agent/deepchat/loop/deepChatLoopEngine' import { InputPreparationCoordinator } from '@/agent/deepchat/loop/inputPreparationCoordinator' import { DeepChatContextCoordinator } from '@/agent/deepchat/loop/contextCoordinator' import type { @@ -142,7 +144,7 @@ import { } from './tapeViewManifest' import { PendingInputCoordinator } from '@/agent/deepchat/pending/pendingInputCoordinator' import { DeepChatPendingInputStore } from '@/agent/deepchat/pending/pendingInputStore' -import { processStream } from './process' +import { MAX_TOOL_CALLS_SKIPPED_ERROR, processStream } from './process' import { cloneBlocksForRenderer } from './echo' import { DeepChatSessionStore, type SessionSummaryState } from './sessionStore' import type { MemoryRuntimePort } from '../memoryPresenter/injection' @@ -178,6 +180,8 @@ import type { SessionUiPort } from '../runtimePorts' import { publishDeepchatEvent } from '@/routes/publishDeepchatEvent' +import { parseMessageMetadata } from '../usageStats' +import { awaitWithAbort } from '@/lib/awaitWithAbort' import { extractToolCallImagePreviews } from '@/lib/toolCallImagePreviews' import { buildAssistantDeliverySegments, @@ -253,7 +257,43 @@ const AUTO_APPROVE_REVIEW_MAX_CONTENT_CHARS = 2_000 const AUTO_APPROVE_REVIEW_TIMEOUT_MS = 30_000 function normalizePermissionMode(mode: PermissionMode | null | undefined): PermissionMode { - return mode === 'default' || mode === 'auto_approve' ? mode : 'full_access' + return mode === 'auto_approve' || mode === 'full_access' ? mode : 'default' +} + +function incrementToolCallAccounting(metadata: MessageMetadata): MessageMetadata { + const currentToolCalls = + typeof metadata.toolCalls === 'number' && + Number.isFinite(metadata.toolCalls) && + metadata.toolCalls >= 0 + ? Math.floor(metadata.toolCalls) + : 0 + return { ...metadata, toolCalls: currentToolCalls + 1 } +} + +function stampTerminalMetadata( + metadata: MessageMetadata, + runOutcome: 'completed' | 'aborted' | 'error', + runStopReason: string, + runId?: string +): MessageMetadata { + return { ...metadata, ...(runId ? { runId } : {}), runOutcome, runStopReason } +} + +function buildUsageFromMetadata(metadata: MessageMetadata): Record | undefined { + const usage: Record = {} + for (const key of [ + 'totalTokens', + 'inputTokens', + 'outputTokens', + 'cachedInputTokens', + 'cacheWriteInputTokens' + ] as const) { + const value = metadata[key] + if (typeof value === 'number' && Number.isFinite(value) && value >= 0) { + usage[key] = value + } + } + return Object.keys(usage).length > 0 ? usage : undefined } function stableStringify(value: unknown): string { @@ -351,6 +391,15 @@ function normalizeReviewDecision(rawText: string, actionHash: string): ToolPermi } } + if (!riskLevel) { + return { + decision: 'ask_user', + userAuthorization, + rationale: 'Auto-review returned an invalid risk level.', + actionHash + } + } + let decision: ToolPermissionReviewResult['decision'] if (rawDecision === 'auto_allow' || rawDecision === 'allow') { decision = 'auto_allow' @@ -537,8 +586,14 @@ type ProviderPermissionInteractionInput = { requestId: string permissionType: 'read' | 'write' | 'all' | 'command' granted: boolean + ownerRun?: LoopRun + signal?: AbortSignal } +type ProviderPermissionProjection = + | { status: 'resolved'; granted: boolean } + | { status: 'error'; message: string } + type PersistedSessionGenerationRow = { provider_id: string model_id: string @@ -672,6 +727,16 @@ export class AgentRuntimePresenter { this.configPresenter = configPresenter this.sqlitePresenter = sqlitePresenter this.toolPresenter = toolPresenter ?? null + this.hookNotificationObserver = hookNotificationObserver + this.providerCatalogPort = runtimePorts?.providerCatalogPort ?? { + getProviderModels: (providerId) => this.configPresenter.getProviderModels?.(providerId) ?? [], + getCustomModels: (providerId) => this.configPresenter.getCustomModels?.(providerId) ?? [] + } + this.sessionPermissionPort = runtimePorts?.sessionPermissionPort + this.acpAsLlmProviderPermission = runtimePorts?.acpAsLlmProviderPermission + this.sessionUiPort = runtimePorts?.sessionUiPort + this.cacheImage = runtimePorts?.cacheImage + this.skillPresenter = runtimePorts?.skillPresenter this.sessionStore = new DeepChatSessionStore(sqlitePresenter) this.messageStore = new DeepChatMessageStore(sqlitePresenter) this.tapeService = new DeepChatTapeService(sqlitePresenter) @@ -753,17 +818,6 @@ export class AgentRuntimePresenter { abortSignal: tool.signal }) }) - this.hookNotificationObserver = hookNotificationObserver - this.providerCatalogPort = runtimePorts?.providerCatalogPort ?? { - getProviderModels: (providerId) => this.configPresenter.getProviderModels?.(providerId) ?? [], - getCustomModels: (providerId) => this.configPresenter.getCustomModels?.(providerId) ?? [] - } - this.sessionPermissionPort = runtimePorts?.sessionPermissionPort - this.acpAsLlmProviderPermission = runtimePorts?.acpAsLlmProviderPermission - this.sessionUiPort = runtimePorts?.sessionUiPort - this.cacheImage = runtimePorts?.cacheImage - this.skillPresenter = runtimePorts?.skillPresenter - const recovered = this.messageStore.recoverPendingMessages() if (recovered > 0) { logger.info(`DeepChatAgent: recovered ${recovered} pending messages to error status`) @@ -821,14 +875,14 @@ export class AgentRuntimePresenter { promptResources: { resolve: async ({ content, scope, workdir, signal }) => { this.throwIfAbortRequested(signal) - const state = await this.getSessionState(sessionId) + const state = await awaitWithAbort(this.getSessionState(sessionId), signal) if (!state) throw new Error(`Session ${sessionId} not found`) const resourceInstance = this.getDeepChatInstance(sessionId) resourceInstance.setAgentId(session.descriptor.id) resourceInstance.setProjectDir(workdir) - const generationSettings = await this.getEffectiveSessionGenerationSettings( - sessionId, - resourceInstance + const generationSettings = await awaitWithAbort( + this.getEffectiveSessionGenerationSettings(sessionId, resourceInstance), + signal ) const normalizedInput = this.normalizeUserMessageInput(content) resourceInstance.replaceRuntimeActivatedSkills(normalizedInput.activeSkills ?? []) @@ -836,26 +890,32 @@ export class AgentRuntimePresenter { let tools: MCPToolDefinition[] = [] let systemPrompt = '' if (scope === 'regular') { - const sessionSkills = await this.resolveActiveSkillNamesForToolProfile( - sessionId, - resourceInstance + const sessionSkills = await awaitWithAbort( + this.resolveActiveSkillNamesForToolProfile(sessionId, resourceInstance), + signal ) const activeSkills = this.resolveEffectiveActiveSkillNames( sessionSkills, resourceInstance ) - tools = await this.loadToolDefinitionsForSession( - sessionId, - workdir, - activeSkills, - resourceInstance + tools = await awaitWithAbort( + this.loadToolDefinitionsForSession( + sessionId, + workdir, + activeSkills, + resourceInstance + ), + signal ) - systemPrompt = await this.buildSystemPromptWithSkills( - sessionId, - generationSettings.systemPrompt, - tools, - activeSkills, - resourceInstance + systemPrompt = await awaitWithAbort( + this.buildSystemPromptWithSkills( + sessionId, + generationSettings.systemPrompt, + tools, + activeSkills, + resourceInstance + ), + signal ) } @@ -1201,7 +1261,7 @@ export class AgentRuntimePresenter { const projectDir = this.normalizeProjectDir(config.projectDir) const permissionMode = normalizePermissionMode(config.permissionMode) logger.info( - `[DeepChatAgent] initSession id=${sessionId} provider=${config.providerId} model=${config.modelId} permission=${permissionMode} projectDir=${projectDir ?? ''}` + `[DeepChatAgent] initSession id=${sessionId} provider=${config.providerId} model=${config.modelId} permission=${permissionMode} hasProjectDir=${projectDir !== null}` ) const generationSettings = await this.sanitizeGenerationSettings( config.providerId, @@ -1265,13 +1325,13 @@ export class AgentRuntimePresenter { const state = instance.getRuntimeState() if (state) { this.getSessionAgentId(sessionId) - if (this.hasPendingInteractions(sessionId)) { - state.status = 'generating' - } if (hydrationMode === 'full') { await this.getEffectiveSessionGenerationSettings(sessionId) } - return { ...state } + return { + ...state, + ...(this.hasPendingInteractions(sessionId) ? { status: 'generating' as const } : {}) + } } const dbSession = this.sessionStore.get(sessionId) as PersistedSessionGenerationRow | undefined @@ -1281,8 +1341,9 @@ export class AgentRuntimePresenter { } this.getSessionAgentId(sessionId) + const hasPendingInteractions = this.hasPendingInteractions(sessionId) const rebuilt: DeepChatSessionState = { - status: this.hasPendingInteractions(sessionId) ? 'generating' : 'idle', + status: 'idle', providerId: dbSession.provider_id, modelId: dbSession.model_id, permissionMode: normalizePermissionMode(dbSession.permission_mode) @@ -1291,7 +1352,10 @@ export class AgentRuntimePresenter { if (hydrationMode === 'full') { await this.getEffectiveSessionGenerationSettings(sessionId) } - return { ...rebuilt } + return { + ...rebuilt, + ...(hasPendingInteractions ? { status: 'generating' as const } : {}) + } } async listPendingInputs(sessionId: string): Promise { @@ -1519,7 +1583,7 @@ export class AgentRuntimePresenter { const supportsAudioInput = this.supportsAudioInput(state.providerId, state.modelId) const projectDir = this.resolveProjectDir(sessionId, context?.projectDir, instance) logger.info( - `[DeepChatAgent] processMessage session=${sessionId} content="${normalizedInput.text.slice(0, 60)}" projectDir=${projectDir ?? ''}` + `[DeepChatAgent] processMessage session=${sessionId} promptLength=${normalizedInput.text.length} fileCount=${normalizedInput.files?.length ?? 0} hasProjectDir=${projectDir !== null}` ) this.setSessionStatus(sessionId, 'generating') @@ -1535,9 +1599,9 @@ export class AgentRuntimePresenter { const preStreamStartedAt = Date.now() this.throwIfAbortRequested(preStreamAbortSignal) let stepStartedAt = Date.now() - const generationSettings = await this.getEffectiveSessionGenerationSettings( - sessionId, - instance + const generationSettings = await awaitWithAbort( + this.getEffectiveSessionGenerationSettings(sessionId, instance), + preStreamAbortSignal ) this.logSlowPreStreamStep(sessionId, 'generation-settings', stepStartedAt) const modelConfig = this.configPresenter.getModelConfig(state.modelId, state.providerId) @@ -1561,9 +1625,9 @@ export class AgentRuntimePresenter { const maxTokens = capAgentRequestMaxTokens(generationSettings.maxTokens, contextBudgetLength) stepStartedAt = Date.now() instance.replaceRuntimeActivatedSkills(normalizedInput.activeSkills ?? []) - const sessionActiveSkillNames = await this.resolveActiveSkillNamesForToolProfile( - sessionId, - instance + const sessionActiveSkillNames = await awaitWithAbort( + this.resolveActiveSkillNamesForToolProfile(sessionId, instance), + preStreamAbortSignal ) this.throwIfStaleDeepChatInstance(sessionId, instance) const effectiveActiveSkillNames = this.resolveEffectiveActiveSkillNames( @@ -1572,23 +1636,29 @@ export class AgentRuntimePresenter { ) this.logSlowPreStreamStep(sessionId, 'active-skills', stepStartedAt) stepStartedAt = Date.now() - const tools = await this.loadToolDefinitionsForSession( - sessionId, - projectDir, - effectiveActiveSkillNames, - instance + const tools = await awaitWithAbort( + this.loadToolDefinitionsForSession( + sessionId, + projectDir, + effectiveActiveSkillNames, + instance + ), + preStreamAbortSignal ) this.logSlowPreStreamStep(sessionId, 'tool-definitions', stepStartedAt) const toolReserveTokens = estimateToolReserveTokens(tools) this.throwIfAbortRequested(preStreamAbortSignal) stepStartedAt = Date.now() const basePromptAssembler = this.createBasePromptAssembler(instance) - const baseSystemPrompt = await basePromptAssembler.assemble({ - sessionId: toAppSessionId(sessionId), - configuredPrompt: generationSettings.systemPrompt, - toolDefinitions: tools, - activeSkillNames: effectiveActiveSkillNames - }) + const baseSystemPrompt = await awaitWithAbort( + basePromptAssembler.assemble({ + sessionId: toAppSessionId(sessionId), + configuredPrompt: generationSettings.systemPrompt, + toolDefinitions: tools, + activeSkillNames: effectiveActiveSkillNames + }), + preStreamAbortSignal + ) this.logSlowPreStreamStep(sessionId, 'system-prompt', stepStartedAt) this.throwIfAbortRequested(preStreamAbortSignal) const userContent: UserMessageContent = { @@ -1700,14 +1770,17 @@ export class AgentRuntimePresenter { stepStartedAt = Date.now() const preparedContext = await this.contextCoordinator.assemble({ assemblePostCompactionPrompt: async () => { - const systemPrompt = await this.postCompactionPromptAssembler.assemble({ - memorySession: instance.getMemorySessionHandle(), - basePrompt: baseSystemPrompt, - summaryText: summaryState.summaryText, - reconstructionAnchor: this.sessionStore.getReconstructionAnchorPromptState(sessionId), - memoryQuery: normalizedInput.text, - memoryMessageId: userMessageId - }) + const systemPrompt = await awaitWithAbort( + this.postCompactionPromptAssembler.assemble({ + memorySession: instance.getMemorySessionHandle(), + basePrompt: baseSystemPrompt, + summaryText: summaryState.summaryText, + reconstructionAnchor: this.sessionStore.getReconstructionAnchorPromptState(sessionId), + memoryQuery: normalizedInput.text, + memoryMessageId: userMessageId + }), + preStreamAbortSignal + ) this.logSlowPreStreamStep(sessionId, 'memory-injection', stepStartedAt) stepStartedAt = Date.now() return systemPrompt @@ -1764,6 +1837,7 @@ export class AgentRuntimePresenter { tools, baseSystemPrompt, resourceInstance: instance, + abortController: preStreamAbortController, maxProviderRounds: context?.maxProviderRounds, refreshSystemPrompt: async (activeSkillNames, refreshedTools) => { const refreshedBasePrompt = await basePromptAssembler.assemble({ @@ -1838,9 +1912,8 @@ export class AgentRuntimePresenter { if (result?.status === 'completed') { void this.drainPendingQueueIfPossible(sessionId, 'completed') } else if (result?.status === 'aborted') { - // Return-path abort: applyProcessResultStatus already dispatched terminal hooks + idle (guarded - // by active run). Append the canceled block, then continue the queue with the next item. - this.writeCanceledTerminalBlock(sessionId, assistantMessageId) + // processStream owns terminal persistence once streaming starts. The lifecycle layer only + // projects hooks/status and advances queued input after the returned abort. void this.drainPendingQueueIfPossible(sessionId, 'completed') } if (result) { @@ -1905,7 +1978,23 @@ export class AgentRuntimePresenter { this.emitMessageRefresh(sessionId, userMessageId) } this.clearSessionAbortController(sessionId, preStreamAbortController) - this.settleAbortedTurn(sessionId, assistantMessageId, streamRunId) + const abortMetadata = stampTerminalMetadata( + { + ...(streamRunId ? { runId: streamRunId } : {}), + provider: state.providerId, + model: state.modelId, + providerRounds: 0, + toolCalls: 0 + }, + 'aborted', + 'user_stop' + ) + this.settleAbortedTurn( + sessionId, + assistantMessageId, + streamRunId, + JSON.stringify(abortMetadata) + ) // Stop/steer: continue the queue automatically with the next item (steer items first). void this.drainPendingQueueIfPossible(sessionId, 'completed') return { @@ -1914,27 +2003,51 @@ export class AgentRuntimePresenter { } } const errorMessage = err instanceof Error ? err.message : String(err) + const stopReason = isContextWindowErrorLike(err) ? 'context_window' : 'pre_stream_error' + const terminalMetadata = stampTerminalMetadata( + { + ...(streamRunId ? { runId: streamRunId } : {}), + provider: state.providerId, + model: state.modelId, + providerRounds: 0, + toolCalls: 0 + }, + 'error', + stopReason + ) if (assistantMessageId) { const existingAssistant = this.messageStore.getMessage(assistantMessageId) const blocks = buildTerminalErrorBlocks( existingAssistant ? this.parseAssistantBlocks(existingAssistant.content) : [], errorMessage ) - this.messageStore.setMessageError(assistantMessageId, blocks) + this.messageStore.setMessageError( + assistantMessageId, + blocks, + JSON.stringify(terminalMetadata) + ) this.emitMessageRefresh(sessionId, assistantMessageId) + publishDeepchatEvent('chat.stream.failed', { + requestId: this.resolveStreamRequestId(sessionId, assistantMessageId), + sessionId, + messageId: assistantMessageId, + failedAt: Date.now(), + error: errorMessage + }) } this.dispatchHook('Stop', { sessionId, providerId: state.providerId, modelId: state.modelId, projectDir, - stop: { reason: 'error', userStop: false } + stop: { reason: stopReason, userStop: false } }) this.dispatchHook('SessionEnd', { sessionId, providerId: state.providerId, modelId: state.modelId, projectDir, + usage: buildUsageFromMetadata(terminalMetadata) ?? null, error: { message: errorMessage } }) this.setSessionStatus(sessionId, 'error') @@ -2148,7 +2261,21 @@ export class AgentRuntimePresenter { return { resumed: false } } + const interactionOwnerRun = instance.getActiveGeneration() + const interactionOwnedByActiveRun = interactionOwnerRun?.messageId === messageId + let interactionAbortController: AbortController | null = null + let interactionAbortSignal: AbortSignal | undefined try { + if (interactionOwnedByActiveRun && interactionOwnerRun.abortController.signal.aborted) { + return { resumed: false } + } + if (interactionOwnedByActiveRun) { + interactionAbortSignal = interactionOwnerRun.abortController.signal + } else if (!interactionOwnerRun) { + interactionAbortController = this.ensureSessionAbortController(sessionId) + interactionAbortSignal = interactionAbortController.signal + } + this.throwIfAbortRequested(interactionAbortSignal) const message = await this.messageStore.getMessage(messageId) if (!message || message.role !== 'assistant') { throw new Error(`Assistant message not found: ${messageId}`) @@ -2179,6 +2306,8 @@ export class AgentRuntimePresenter { let waitingForUserMessage = false let resumeBudgetToolCall: ResumeBudgetToolCall | null = null let emitResolvedToolHook: (() => void) | null = null + let resumeAccounting = parseMessageMetadata(message.metadata) + let accountingChanged = false const actionBlock = blocks[currentEntry.blockIndex] const toolCall = actionBlock.tool_call if (!toolCall?.id) { @@ -2191,13 +2320,16 @@ export class AgentRuntimePresenter { } if (this.isSkillDraftConfirmationBlock(actionBlock)) { - const result = await this.handleSkillDraftInteraction( - sessionId, - instance, - blocks, - actionBlock, - toolCall, - response + const result = await awaitWithAbort( + this.handleSkillDraftInteraction( + sessionId, + instance, + blocks, + actionBlock, + toolCall, + response + ), + interactionAbortSignal ) if (!this.isCurrentDeepChatInstance(sessionId, instance)) { return { resumed: false } @@ -2213,7 +2345,7 @@ export class AgentRuntimePresenter { instance.advancePendingToolBatch({ committedResultCallId: toolCall.id }) } else if (response.kind === 'question_other') { const deferredResult = 'User chose to answer with a follow-up message.' - this.markQuestionResolved(actionBlock, '') + this.markQuestionResolved(actionBlock, '', true) this.updateToolCallResponse(blocks, toolCall.id, deferredResult, false) instance.advancePendingToolBatch({ committedResultCallId: toolCall.id }) waitingForUserMessage = true @@ -2237,14 +2369,19 @@ export class AgentRuntimePresenter { const requestId = permissionPayload?.requestId?.trim() const providerId = permissionPayload?.providerId?.trim() if (providerId === 'acp' && requestId) { - await this.resolveProviderPermissionInteraction({ - sessionId, - messageId, - toolCallId: toolCall.id, - requestId, - permissionType, - granted: response.granted - }) + await awaitWithAbort( + this.resolveProviderPermissionInteraction({ + sessionId, + messageId, + toolCallId: toolCall.id, + requestId, + permissionType, + granted: response.granted, + ownerRun: interactionOwnerRun, + signal: interactionAbortSignal + }), + interactionAbortSignal + ) return { resumed: false } } const state = this.getDeepChatRuntimeState(sessionId) @@ -2253,24 +2390,55 @@ export class AgentRuntimePresenter { if (response.granted) { this.markPermissionResolved(actionBlock, true, permissionType) - await this.grantPermissionForPayload(sessionId, permissionPayload, toolCall) - this.dispatchHook('PreToolUse', { - sessionId, - messageId, - providerId: state?.providerId, - modelId: state?.modelId, - projectDir, - tool: { - callId: toolCall.id, - name: toolCall.name, - params: toolCall.params + await awaitWithAbort( + this.grantPermissionForPayload(sessionId, permissionPayload, toolCall), + interactionAbortSignal + ) + const nextToolCallAccounting = incrementToolCallAccounting(resumeAccounting) + let deferredToolCallCounted = false + const markDeferredToolCallStarted = () => { + if (deferredToolCallCounted) { + return } - }) - const execution = await this.executeDeferredToolCall(sessionId, messageId, toolCall) + deferredToolCallCounted = true + resumeAccounting = nextToolCallAccounting + accountingChanged = true + this.messageStore.updateAssistantMetadata(messageId, JSON.stringify(resumeAccounting)) + } + let execution: DeferredToolExecutionResult + if ((nextToolCallAccounting.toolCalls ?? 0) > MAX_TOOL_CALLS) { + execution = { + responseText: MAX_TOOL_CALLS_SKIPPED_ERROR, + isError: true + } + } else { + this.dispatchHook('PreToolUse', { + sessionId, + messageId, + providerId: state?.providerId, + modelId: state?.modelId, + projectDir, + tool: { + callId: toolCall.id, + name: toolCall.name, + params: toolCall.params + } + }) + execution = await this.executeDeferredToolCall( + sessionId, + messageId, + toolCall, + markDeferredToolCallStarted + ) + if ((execution.invoked || execution.terminalError) && !deferredToolCallCounted) { + markDeferredToolCallStarted() + } + } if (execution.invoked) { instance.advancePendingToolBatch({ invokedCallId: toolCall.id }) } if (execution.terminalError) { + const terminalMetadata = stampTerminalMetadata(resumeAccounting, 'error', 'tool_error') instance.advancePendingToolBatch({ committedResultCallId: toolCall.id }) this.dispatchHook('PostToolUseFailure', { sessionId, @@ -2286,7 +2454,7 @@ export class AgentRuntimePresenter { } }) this.updateToolCallResponse(blocks, toolCall.id, execution.terminalError, true) - this.messageStore.setMessageError(messageId, blocks) + this.messageStore.setMessageError(messageId, blocks, JSON.stringify(terminalMetadata)) this.emitMessageRefresh(sessionId, messageId) publishDeepchatEvent('chat.stream.failed', { requestId: this.resolveStreamRequestId(sessionId, messageId), @@ -2301,7 +2469,7 @@ export class AgentRuntimePresenter { providerId: state?.providerId, modelId: state?.modelId, projectDir, - stop: { reason: 'error', userStop: false } + stop: { reason: 'tool_error', userStop: false } }) this.dispatchHook('SessionEnd', { sessionId, @@ -2309,6 +2477,7 @@ export class AgentRuntimePresenter { providerId: state?.providerId, modelId: state?.modelId, projectDir, + usage: buildUsageFromMetadata(terminalMetadata) ?? null, error: { message: execution.terminalError } }) this.setSessionStatus(sessionId, 'error') @@ -2404,11 +2573,20 @@ export class AgentRuntimePresenter { throw new Error(`Unsupported action type: ${actionBlock.action_type}`) } - this.messageStore.updateAssistantContent(messageId, blocks) const remainingPending = this.reconcilePendingInteractionEntries( instance, this.collectPendingInteractionEntries(messageId, blocks) ) + const awaitsUserFollowUp = waitingForUserMessage || this.hasQuestionFollowUpIntent(blocks) + const finishesForUserFollowUp = awaitsUserFollowUp && remainingPending.length === 0 + const persistedMetadata = finishesForUserFollowUp + ? stampTerminalMetadata(resumeAccounting, 'completed', 'user_follow_up') + : resumeAccounting + this.messageStore.updateAssistantContent( + messageId, + blocks, + finishesForUserFollowUp || accountingChanged ? JSON.stringify(persistedMetadata) : undefined + ) this.replacePendingInteractions(instance, remainingPending) this.emitMessageRefresh(sessionId, messageId) @@ -2419,22 +2597,54 @@ export class AgentRuntimePresenter { return { resumed: false } } - if (waitingForUserMessage) { + if (awaitsUserFollowUp) { emitResolvedToolHook?.() this.messageStore.updateMessageStatus(messageId, 'sent') + this.dispatchTerminalHooks(sessionId, this.getDeepChatRuntimeState(sessionId), { + status: 'completed', + stopReason: 'user_follow_up', + usage: buildUsageFromMetadata(persistedMetadata) + }) this.setSessionStatus(sessionId, 'idle') return { resumed: false, waitingForUserMessage: true } } + this.clearSessionAbortController(sessionId, interactionAbortController ?? undefined) const resumed = await this.resumeAssistantMessage( sessionId, messageId, blocks, - resumeBudgetToolCall + resumeBudgetToolCall, + resumeAccounting ) emitResolvedToolHook?.() return { resumed } + } catch (error) { + if (this.isAbortError(error) || interactionAbortSignal?.aborted) { + if (interactionOwnedByActiveRun) { + return { resumed: false } + } + const accounting = parseMessageMetadata( + this.messageStore.getMessage(messageId)?.metadata ?? '{}' + ) + if (interactionAbortController) { + this.clearSessionAbortController(sessionId, interactionAbortController) + } + instance.replacePendingInteractions([]) + this.settleAbortedTurn( + sessionId, + messageId, + undefined, + JSON.stringify(stampTerminalMetadata(accounting, 'aborted', 'user_stop')) + ) + void this.drainPendingQueueIfPossible(sessionId, 'completed') + return { resumed: false } + } + throw error } finally { + if (interactionAbortController) { + this.clearSessionAbortController(sessionId, interactionAbortController) + } instance.unlockInteraction(messageId, toolCallId) } } @@ -2442,10 +2652,10 @@ export class AgentRuntimePresenter { async setPermissionMode(sessionId: string, mode: PermissionMode): Promise { const normalizedMode = normalizePermissionMode(mode) const state = this.getDeepChatRuntimeState(sessionId) + this.sessionStore.updatePermissionMode(sessionId, normalizedMode) if (state) { state.permissionMode = normalizedMode } - this.sessionStore.updatePermissionMode(sessionId, normalizedMode) } async setSessionModel(sessionId: string, providerId: string, modelId: string): Promise { @@ -2470,24 +2680,26 @@ export class AgentRuntimePresenter { systemPrompt: currentGeneration.systemPrompt }) + this.sessionStore.updateSessionConfiguration( + sessionId, + nextProviderId, + nextModelId, + this.buildPersistedGenerationSettingsReplacement(sanitized) + ) + + const instance = this.getDeepChatInstance(sessionId) if (state) { state.providerId = nextProviderId state.modelId = nextModelId } else { - this.getDeepChatInstance(sessionId).setRuntimeState({ + instance.setRuntimeState({ status: 'idle', providerId: nextProviderId, modelId: nextModelId, permissionMode: normalizePermissionMode(dbSession?.permission_mode) }) } - - this.sessionStore.updateSessionModel(sessionId, nextProviderId, nextModelId) - this.sessionStore.updateGenerationSettings( - sessionId, - this.buildPersistedGenerationSettingsReplacement(sanitized) - ) - this.getDeepChatInstance(sessionId).setGenerationSettings(sanitized) + instance.setGenerationSettings(sanitized) this.invalidateSystemPromptCache(sessionId) this.invalidateToolProfileCache(sessionId) } @@ -2520,6 +2732,14 @@ export class AgentRuntimePresenter { config.generationSettings ?? {} ) + this.sessionStore.updateSessionConfiguration( + sessionId, + nextProviderId, + nextModelId, + this.buildPersistedGenerationSettingsReplacement(sanitizedGenerationSettings), + permissionMode + ) + const instance = this.getDeepChatInstance(sessionId) instance.setRuntimeState({ status: state?.status ?? 'idle', @@ -2527,12 +2747,6 @@ export class AgentRuntimePresenter { modelId: nextModelId, permissionMode }) - this.sessionStore.updateSessionModel(sessionId, nextProviderId, nextModelId) - this.sessionStore.updatePermissionMode(sessionId, permissionMode) - this.sessionStore.updateGenerationSettings( - sessionId, - this.buildPersistedGenerationSettingsReplacement(sanitizedGenerationSettings) - ) instance.setAgentId(nextAgentId) instance.setProjectDir(this.normalizeProjectDir(config.projectDir)) instance.setGenerationSettings(sanitizedGenerationSettings) @@ -2588,11 +2802,11 @@ export class AgentRuntimePresenter { const current = await this.getEffectiveSessionGenerationSettings(sessionId) const sanitized = await this.sanitizeGenerationSettings(providerId, modelId, settings, current) - this.getDeepChatInstance(sessionId).setGenerationSettings(sanitized) this.sessionStore.updateGenerationSettings( sessionId, this.buildPersistedGenerationSettingsPatch(settings, sanitized) ) + this.getDeepChatInstance(sessionId).setGenerationSettings(sanitized) if (Object.prototype.hasOwnProperty.call(settings, 'systemPrompt')) { this.invalidateSystemPromptCache(sessionId) } @@ -2600,21 +2814,52 @@ export class AgentRuntimePresenter { } async cancelGeneration(sessionId: string): Promise { - // Single responsibility: request the abort and release controllers/permissions. Terminal - // settlement (canceled block + Stop/SessionEnd hooks + idle status + queue drain) is owned by the - // in-flight processMessage / resumeAssistantMessage handler, which always observes the abort and - // settles exactly once. cancelGeneration deliberately does NOT clear the active generation, write - // the terminal block, dispatch hooks, or set status. - this.getHydratedDeepChatInstance(sessionId)?.requestGenerationAbort() + const instance = this.getHydratedDeepChatInstance(sessionId) + if (!instance) { + return + } + + if (!instance.hasPendingInteractions()) { + this.refreshPendingInteractionsFromStore(sessionId) + } + const pendingInteractions = instance.getPendingInteractions() + const hasDeferredHandler = pendingInteractions.some((interaction) => + instance.hasDeferredToolAbortController(interaction.toolCallId) + ) + const hasAsyncSettlementOwner = Boolean( + instance.getActiveGeneration() || instance.getAbortController() || hasDeferredHandler + ) + + instance.requestGenerationAbort() this.abortDeferredToolAbortControllers(sessionId) this.clearActiveProviderPermissionsForSession(sessionId) + + if (hasAsyncSettlementOwner || pendingInteractions.length === 0) { + return + } + + const messageId = pendingInteractions[0].messageId + const metadata = parseMessageMetadata(this.messageStore.getMessage(messageId)?.metadata ?? '{}') + const terminalMetadata = stampTerminalMetadata(metadata, 'aborted', 'user_stop') + instance.replacePendingInteractions([]) + this.settleAbortedTurn( + sessionId, + messageId, + terminalMetadata.runId, + JSON.stringify(terminalMetadata) + ) + void this.drainPendingQueueIfPossible(sessionId, 'completed') } /** * Append the canceled terminal block to an assistant message after a stop/steer abort. Idempotent * via buildTerminalErrorBlocks (won't duplicate the block). */ - private writeCanceledTerminalBlock(sessionId: string, messageId: string | null): void { + private writeCanceledTerminalBlock( + sessionId: string, + messageId: string | null, + metadata?: string + ): void { if (!messageId) { return } @@ -2626,7 +2871,7 @@ export class AgentRuntimePresenter { this.parseAssistantBlocks(assistantMessage.content), 'common.error.userCanceledGeneration' ) - this.messageStore.setMessageError(messageId, blocks) + this.messageStore.setMessageError(messageId, blocks, metadata) this.emitMessageRefresh(sessionId, messageId) } @@ -2635,12 +2880,19 @@ export class AgentRuntimePresenter { * terminal block + terminal hooks + idle status. The return-path settles via applyProcessResultStatus * instead. The caller remains responsible for draining the queue. */ - private settleAbortedTurn(sessionId: string, messageId: string | null, runId?: string): void { - this.writeCanceledTerminalBlock(sessionId, messageId) + private settleAbortedTurn( + sessionId: string, + messageId: string | null, + runId?: string, + metadata?: string + ): void { + this.writeCanceledTerminalBlock(sessionId, messageId, metadata) + const usage = metadata ? buildUsageFromMetadata(parseMessageMetadata(metadata)) : undefined this.dispatchTerminalHooks(sessionId, this.getDeepChatRuntimeState(sessionId), { status: 'aborted', stopReason: 'user_stop', - errorMessage: 'common.error.userCanceledGeneration' + errorMessage: 'common.error.userCanceledGeneration', + usage }) const instance = this.getHydratedDeepChatInstance(sessionId) const activeGeneration = instance?.getActiveGeneration() @@ -3131,10 +3383,13 @@ export class AgentRuntimePresenter { } this.setSessionStatusForInstance(sessionId, instance, 'generating') + const compactionAbortController = this.ensureSessionAbortController(sessionId) + const compactionAbortSignal = compactionAbortController.signal try { - const generationSettings = await this.getEffectiveSessionGenerationSettings( - sessionId, - instance + this.throwIfAbortRequested(compactionAbortSignal) + const generationSettings = await awaitWithAbort( + this.getEffectiveSessionGenerationSettings(sessionId, instance), + compactionAbortSignal ) const interleavedReasoning = this.resolveInterleavedReasoningConfig( state.providerId, @@ -3148,22 +3403,27 @@ export class AgentRuntimePresenter { state.modelId ) const maxTokens = capAgentRequestMaxTokens(generationSettings.maxTokens, contextBudgetLength) - const activeSkillNames = await this.resolveActiveSkillNamesForToolProfile(sessionId, instance) + const activeSkillNames = await awaitWithAbort( + this.resolveActiveSkillNamesForToolProfile(sessionId, instance), + compactionAbortSignal + ) this.throwIfStaleDeepChatInstance(sessionId, instance) const projectDir = this.resolveProjectDir(sessionId, undefined, instance) - const tools = await this.loadToolDefinitionsForSession( - sessionId, - projectDir, - activeSkillNames, - instance + const tools = await awaitWithAbort( + this.loadToolDefinitionsForSession(sessionId, projectDir, activeSkillNames, instance), + compactionAbortSignal ) const toolReserveTokens = estimateToolReserveTokens(tools) - const baseSystemPrompt = await this.createBasePromptAssembler(instance).assemble({ - sessionId: toAppSessionId(sessionId), - configuredPrompt: generationSettings.systemPrompt, - toolDefinitions: tools, - activeSkillNames - }) + const baseSystemPrompt = await awaitWithAbort( + this.createBasePromptAssembler(instance).assemble({ + sessionId: toAppSessionId(sessionId), + configuredPrompt: generationSettings.systemPrompt, + toolDefinitions: tools, + activeSkillNames + }), + compactionAbortSignal + ) + this.throwIfAbortRequested(compactionAbortSignal) const tapeReady = this.tapeService.ensureSessionTapeReady(sessionId, this.messageStore) const intent = await this.compactionService.prepareForManualCompaction({ @@ -3179,8 +3439,10 @@ export class AgentRuntimePresenter { preserveInterleavedReasoning: interleavedReasoning.preserveReasoningContent, preserveEmptyInterleavedReasoning: interleavedReasoning.preserveEmptyReasoningContent === true, - historyRecords: tapeReady.historyRecords + historyRecords: tapeReady.historyRecords, + signal: compactionAbortSignal }) + this.throwIfAbortRequested(compactionAbortSignal) this.throwIfStaleDeepChatInstance(sessionId, instance) if (!intent) { @@ -3190,7 +3452,13 @@ export class AgentRuntimePresenter { } } - const summaryState = await this.applyCompactionIntent(sessionId, intent, undefined, instance) + const summaryState = await this.applyCompactionIntent( + sessionId, + intent, + { signal: compactionAbortSignal }, + instance + ) + this.throwIfAbortRequested(compactionAbortSignal) this.throwIfStaleDeepChatInstance(sessionId, instance) const compacted = summaryState.summaryUpdatedAt !== intent.previousState.summaryUpdatedAt return { @@ -3198,7 +3466,13 @@ export class AgentRuntimePresenter { state: await this.getSessionCompactionStateForInstance(sessionId, instance) } } finally { - this.setSessionStatusForInstance(sessionId, instance, 'idle') + const currentController = instance.getAbortController() + const stillOwnsLifecycle = + currentController === undefined || currentController === compactionAbortController + this.clearSessionAbortController(sessionId, compactionAbortController) + if (stillOwnsLifecycle) { + this.setSessionStatusForInstance(sessionId, instance, 'idle') + } } } @@ -3362,6 +3636,7 @@ export class AgentRuntimePresenter { tools?: MCPToolDefinition[] baseSystemPrompt?: string initialBlocks?: AssistantMessageBlock[] + initialAccounting?: MessageMetadata promptPreview?: string interleavedReasoning?: InterleavedReasoningConfig viewContext?: PendingTapeViewContext @@ -3372,6 +3647,7 @@ export class AgentRuntimePresenter { maxProviderRounds?: number preStreamStartedAt?: number onRunRegistered?: (runId: string) => void + abortController?: AbortController }): Promise<{ runId: string; result: ProcessResult }> { const { sessionId, @@ -3382,16 +3658,21 @@ export class AgentRuntimePresenter { tools: providedTools, baseSystemPrompt, initialBlocks, + initialAccounting, promptPreview, interleavedReasoning: providedInterleavedReasoning, viewContext, refreshSystemPrompt, maxProviderRounds, preStreamStartedAt, - onRunRegistered + onRunRegistered, + abortController: providedAbortController } = args const resourceInstance = providedResourceInstance ?? this.getDeepChatInstance(sessionId) this.throwIfStaleDeepChatInstance(sessionId, resourceInstance) + const abortController = providedAbortController ?? this.ensureSessionAbortController(sessionId) + const abortSignal = abortController.signal + this.throwIfAbortRequested(abortSignal) const state = resourceInstance.getRuntimeState() if (!state) { throw new Error(`Session ${sessionId} not found`) @@ -3415,9 +3696,9 @@ export class AgentRuntimePresenter { } ).getProviderInstance(state.providerId) - const generationSettings = await this.getEffectiveSessionGenerationSettings( - sessionId, - resourceInstance + const generationSettings = await awaitWithAbort( + this.getEffectiveSessionGenerationSettings(sessionId, resourceInstance), + abortSignal ) const baseModelConfig = this.configPresenter.getModelConfig(state.modelId, state.providerId) const interleavedReasoning = @@ -3466,14 +3747,14 @@ export class AgentRuntimePresenter { const temperature = generationSettings.temperature const maxTokens = capAgentRequestMaxTokens(generationSettings.maxTokens, contextBudgetLength) - const streamSessionActiveSkillNames = await this.resolveActiveSkillNamesForToolProfile( - sessionId, - resourceInstance + const streamSessionActiveSkillNames = await awaitWithAbort( + this.resolveActiveSkillNamesForToolProfile(sessionId, resourceInstance), + abortSignal ) this.throwIfStaleDeepChatInstance(sessionId, resourceInstance) - const streamExtensionPolicy = await this.resolveAgentExtensionPolicy( - sessionId, - resourceInstance + const streamExtensionPolicy = await awaitWithAbort( + this.resolveAgentExtensionPolicy(sessionId, resourceInstance), + abortSignal ) this.throwIfStaleDeepChatInstance(sessionId, resourceInstance) const getEffectiveRuntimeSkillNames = (baseSkillNames = streamSessionActiveSkillNames) => @@ -3481,12 +3762,15 @@ export class AgentRuntimePresenter { const toolCatalog = this.createSessionToolCatalogPort(sessionId, projectDir, resourceInstance) const tools = providedTools ?? - (await toolCatalog.resolve({ activeSkillNames: getEffectiveRuntimeSkillNames() })) + (await awaitWithAbort( + toolCatalog.resolve({ activeSkillNames: getEffectiveRuntimeSkillNames() }), + abortSignal + )) this.throwIfStaleDeepChatInstance(sessionId, resourceInstance) const supportsVision = this.supportsVision(state.providerId, state.modelId) const supportsAudioInput = this.supportsAudioInput(state.providerId, state.modelId) - const abortController = new AbortController() + abortController.signal.throwIfAborted() const loopRun = createLoopRun({ runId: `${sessionId}:${++this.nextRunSequence}`, sessionId: toAppSessionId(sessionId), @@ -3576,7 +3860,9 @@ export class AgentRuntimePresenter { requestModelConfig, requestTemperature, requestMaxTokens, - requestTools + requestTools, + onProviderRequestStart, + assertProviderRequestAvailable ) { const requestBypassesContextBudget = shouldBypassContextBudget( state.providerId, @@ -3684,15 +3970,20 @@ export class AgentRuntimePresenter { } }, provider: { + assertAvailable: assertProviderRequestAvailable, stream: ({ messages, modelId, modelConfig, temperature, maxTokens, tools }) => provider.coreStream(messages, modelId, modelConfig, temperature, maxTokens, tools), - beforeStream: logPreStreamBoundary + beforeStream: () => { + onProviderRequestStart?.() + logPreStreamBoundary() + } }, isContextOverflowEvent: isFirstProviderContextOverflowEvent, isContextOverflowError: isContextWindowErrorLike, createAbortError }) }, + coreStreamReportsProviderStart: true, providerId: state.providerId, modelId: state.modelId, modelConfig, @@ -3701,6 +3992,7 @@ export class AgentRuntimePresenter { interleavedReasoning, permissionMode: state.permissionMode, initialBlocks, + initialAccounting, onFirstProviderRoundReady: () => { if ( !abortController.signal.aborted && @@ -4226,7 +4518,8 @@ export class AgentRuntimePresenter { sessionId: string, messageId: string, initialBlocks: AssistantMessageBlock[], - budgetToolCall?: ResumeBudgetToolCall | null + budgetToolCall?: ResumeBudgetToolCall | null, + initialAccounting?: MessageMetadata ): Promise { const instance = this.getDeepChatInstance(sessionId) if (!instance.tryBeginResume(messageId)) { @@ -4235,6 +4528,9 @@ export class AgentRuntimePresenter { let preStreamAbortController: AbortController | null = null let preStreamAbortSignal: AbortSignal | undefined let streamRunId: string | undefined + const resumeAccounting = + initialAccounting ?? + parseMessageMetadata(this.messageStore.getMessage(messageId)?.metadata ?? '{}') try { this.throwIfStaleDeepChatInstance(sessionId, instance) @@ -4247,9 +4543,9 @@ export class AgentRuntimePresenter { preStreamAbortController = this.ensureSessionAbortController(sessionId) preStreamAbortSignal = preStreamAbortController.signal this.throwIfAbortRequested(preStreamAbortSignal) - const generationSettings = await this.getEffectiveSessionGenerationSettings( - sessionId, - instance + const generationSettings = await awaitWithAbort( + this.getEffectiveSessionGenerationSettings(sessionId, instance), + preStreamAbortSignal ) const modelConfig = this.configPresenter.getModelConfig(state.modelId, state.providerId) const useContextBudget = this.shouldUseDeepChatContextBudget( @@ -4271,22 +4567,26 @@ export class AgentRuntimePresenter { ) const maxTokens = capAgentRequestMaxTokens(generationSettings.maxTokens, contextBudgetLength) const projectDir = this.resolveProjectDir(sessionId, undefined, instance) - const activeSkillNames = await this.resolveActiveSkillNamesForToolProfile(sessionId, instance) + const activeSkillNames = await awaitWithAbort( + this.resolveActiveSkillNamesForToolProfile(sessionId, instance), + preStreamAbortSignal + ) this.throwIfStaleDeepChatInstance(sessionId, instance) - const tools = await this.loadToolDefinitionsForSession( - sessionId, - projectDir, - activeSkillNames, - instance + const tools = await awaitWithAbort( + this.loadToolDefinitionsForSession(sessionId, projectDir, activeSkillNames, instance), + preStreamAbortSignal ) const toolReserveTokens = estimateToolReserveTokens(tools) this.throwIfAbortRequested(preStreamAbortSignal) - const baseSystemPrompt = await this.createBasePromptAssembler(instance).assemble({ - sessionId: toAppSessionId(sessionId), - configuredPrompt: generationSettings.systemPrompt, - toolDefinitions: tools, - activeSkillNames - }) + const baseSystemPrompt = await awaitWithAbort( + this.createBasePromptAssembler(instance).assemble({ + sessionId: toAppSessionId(sessionId), + configuredPrompt: generationSettings.systemPrompt, + toolDefinitions: tools, + activeSkillNames + }), + preStreamAbortSignal + ) this.throwIfAbortRequested(preStreamAbortSignal) let resumeTargetOrderSeq: number | undefined const preparedInput = await this.inputPreparationCoordinator.prepareExisting({ @@ -4343,14 +4643,17 @@ export class AgentRuntimePresenter { this.throwIfAbortRequested(preStreamAbortSignal) const preparedContext = await this.contextCoordinator.assemble({ assemblePostCompactionPrompt: async () => - await this.postCompactionPromptAssembler.assemble({ - memorySession: instance.getMemorySessionHandle(), - basePrompt: baseSystemPrompt, - summaryText: summaryState.summaryText, - reconstructionAnchor: this.sessionStore.getReconstructionAnchorPromptState(sessionId), - memoryQuery: this.memoryCoordinator.getLatestUserQuery(sessionId), - memoryMessageId: messageId - }), + await awaitWithAbort( + this.postCompactionPromptAssembler.assemble({ + memorySession: instance.getMemorySessionHandle(), + basePrompt: baseSystemPrompt, + summaryText: summaryState.summaryText, + reconstructionAnchor: this.sessionStore.getReconstructionAnchorPromptState(sessionId), + memoryQuery: this.memoryCoordinator.getLatestUserQuery(sessionId), + memoryMessageId: messageId + }), + preStreamAbortSignal + ), buildView: (systemPrompt) => buildTapeResumeView({ sessionId, @@ -4400,7 +4703,16 @@ export class AgentRuntimePresenter { await this.toolOutputGuard.cleanupOffloadedOutput(budgetToolCall.offloadPath) this.throwIfStaleDeepChatInstance(sessionId, instance) this.updateToolCallResponse(initialBlocks, budgetToolCall.id, resumeBudget.message, true) - this.messageStore.setMessageError(messageId, initialBlocks) + const terminalMetadata = stampTerminalMetadata( + resumeAccounting, + 'error', + 'context_window' + ) + this.messageStore.setMessageError( + messageId, + initialBlocks, + JSON.stringify(terminalMetadata) + ) this.emitMessageRefresh(sessionId, messageId) publishDeepchatEvent('chat.stream.failed', { requestId: this.resolveStreamRequestId(sessionId, messageId), @@ -4409,6 +4721,12 @@ export class AgentRuntimePresenter { failedAt: Date.now(), error: resumeBudget.message }) + this.dispatchTerminalHooks(sessionId, state, { + status: 'error', + stopReason: 'context_window', + errorMessage: resumeBudget.message, + usage: buildUsageFromMetadata(terminalMetadata) + }) this.setSessionStatus(sessionId, 'error') this.memoryIngestionObserver.afterTurnSettled({ session: instance.getMemorySessionHandle(), @@ -4427,9 +4745,12 @@ export class AgentRuntimePresenter { messages: resumeContext, projectDir, resourceInstance: instance, + abortController: preStreamAbortController, tools, baseSystemPrompt, initialBlocks, + initialAccounting: resumeAccounting, + maxProviderRounds: resumeAccounting.maxProviderRounds, interleavedReasoning, viewContext: { taskType: 'resume', @@ -4452,10 +4773,6 @@ export class AgentRuntimePresenter { } finally { this.clearActiveGeneration(sessionId, runId) } - if (result?.status === 'aborted') { - // Return-path abort: applyProcessResultStatus already handled hooks + idle. - this.writeCanceledTerminalBlock(sessionId, messageId) - } if (result?.status === 'completed' || result?.status === 'aborted') { void this.drainPendingQueueIfPossible(sessionId, 'completed') } @@ -4479,15 +4796,42 @@ export class AgentRuntimePresenter { console.error('[DeepChatAgent] resumeAssistantMessage error:', error) if (this.isAbortError(error) || preStreamAbortSignal?.aborted) { this.clearSessionAbortController(sessionId, preStreamAbortController ?? undefined) - this.settleAbortedTurn(sessionId, messageId, streamRunId) + this.settleAbortedTurn( + sessionId, + messageId, + streamRunId, + JSON.stringify( + stampTerminalMetadata(resumeAccounting, 'aborted', 'user_stop', streamRunId) + ) + ) // Stop/steer: continue the queue automatically with the next item (steer items first). void this.drainPendingQueueIfPossible(sessionId, 'completed') return false } const errorMessage = error instanceof Error ? error.message : String(error) + const stopReason = isContextWindowErrorLike(error) ? 'context_window' : 'pre_stream_error' + const terminalMetadata = stampTerminalMetadata( + resumeAccounting, + 'error', + stopReason, + streamRunId + ) const blocks = buildTerminalErrorBlocks(initialBlocks, errorMessage) - this.messageStore.setMessageError(messageId, blocks) + this.messageStore.setMessageError(messageId, blocks, JSON.stringify(terminalMetadata)) this.emitMessageRefresh(sessionId, messageId) + publishDeepchatEvent('chat.stream.failed', { + requestId: this.resolveStreamRequestId(sessionId, messageId), + sessionId, + messageId, + failedAt: Date.now(), + error: errorMessage + }) + this.dispatchTerminalHooks(sessionId, this.getDeepChatRuntimeState(sessionId), { + status: 'error', + stopReason, + errorMessage, + usage: buildUsageFromMetadata(terminalMetadata) + }) this.setSessionStatus(sessionId, 'error') throw error } finally { @@ -6055,45 +6399,132 @@ export class AgentRuntimePresenter { input: ProviderPermissionInteractionInput ): Promise { const instance = this.getHydratedDeepChatInstance(input.sessionId) - const active = instance?.getActiveProviderPermission(input.requestId) - let resolution: { status: 'resolved' } | { status: 'stale'; error: unknown } + const activeCandidate = instance?.getActiveProviderPermission(input.requestId) + const active = + activeCandidate?.messageId === input.messageId && + activeCandidate.toolCallId === input.toolCallId + ? activeCandidate + : undefined + const hasConflictingActive = Boolean(activeCandidate && !active) + const ownerRun = input.ownerRun?.messageId === input.messageId ? input.ownerRun : undefined - try { - resolution = await this.resolveProviderPermissionSafely( - active - ? () => active.resolve(input.granted) - : () => - this.requireAcpAsLlmProviderPermission().resolveAgentPermission( - input.requestId, - input.granted - ) + if (input.signal?.aborted || ownerRun?.abortController.signal.aborted) { + return + } + + if (ownerRun) { + if (hasConflictingActive) { + const projection: ProviderPermissionProjection = { + status: 'error', + message: 'ACP permission request ownership changed.' + } + this.updateActiveProviderPermissionState(ownerRun, input, projection) + this.updatePersistedProviderPermissionState(input, projection) + if (instance) { + this.removePendingProviderPermission(instance, input) + } + return + } + + let resolution: { status: 'resolved' } | { status: 'stale'; error: unknown } + + try { + resolution = await this.resolveProviderPermissionSafely( + active + ? () => active.resolve(input.granted) + : () => + this.requireAcpAsLlmProviderPermission().resolveAgentPermission( + input.requestId, + input.granted + ) + ) + } finally { + instance?.clearActiveProviderPermission(input.requestId, active) + } + + if ( + input.signal?.aborted || + ownerRun.abortController.signal.aborted || + !instance?.isActiveRun(ownerRun.runId) + ) { + return + } + + if (resolution.status === 'stale') { + console.warn( + `[DeepChatAgent] ACP permission request expired while its generation remained active: ${input.requestId}`, + resolution.error + ) + } + + if (!active || resolution.status === 'stale') { + const projection: ProviderPermissionProjection = + resolution.status === 'resolved' + ? { status: 'resolved', granted: input.granted } + : { status: 'error', message: 'Permission request expired.' } + this.updateActiveProviderPermissionState(ownerRun, input, projection) + this.updatePersistedProviderPermissionState(input, projection) + } + + this.removePendingProviderPermission(instance, input) + return + } + + if (hasConflictingActive) { + this.failProviderPermissionInteraction( + input, + 'ACP permission request ownership changed.', + instance ) + return + } + + let resolution: + | { status: 'resolved' } + | { status: 'stale'; error: unknown } + | { status: 'failed'; error: unknown } + + try { + try { + resolution = await this.resolveProviderPermissionSafely( + active + ? () => active.resolve(false) + : () => + this.requireAcpAsLlmProviderPermission().resolveAgentPermission( + input.requestId, + false + ) + ) + } catch (error) { + resolution = { status: 'failed', error } + } } finally { instance?.clearActiveProviderPermission(input.requestId, active) } - if (active && resolution.status === 'resolved') { - this.refreshPendingInteractionsFromStore(input.sessionId) + if (input.signal?.aborted) { return } if (resolution.status === 'stale') { console.warn( - `[DeepChatAgent] Clearing stale ACP permission request ${input.requestId}:`, + `[DeepChatAgent] Failing stale ACP permission request ${input.requestId}:`, + resolution.error + ) + } else if (resolution.status === 'failed') { + console.warn( + `[DeepChatAgent] Failed to deny orphaned ACP permission request ${input.requestId}:`, resolution.error ) } - this.updatePersistedProviderPermissionState( - input.messageId, - input.toolCallId, - input.requestId, - input.permissionType, - resolution.status === 'resolved' ? input.granted : false, - resolution.status === 'stale' ? 'Permission request expired.' : undefined + this.failProviderPermissionInteraction( + input, + resolution.status === 'stale' + ? 'Permission request expired.' + : 'ACP permission request lost its active generation.', + instance ) - this.finishProviderPermissionInteraction(input.sessionId, input.messageId) - this.refreshPendingInteractionsFromStore(input.sessionId) } private async resolveProviderPermissionSafely( @@ -6116,43 +6547,128 @@ export class AgentRuntimePresenter { return Boolean(message?.startsWith('Unknown ACP permission request:')) } - private finishProviderPermissionInteraction(sessionId: string, messageId: string): void { - this.messageStore.updateMessageStatus(messageId, 'sent') - this.setSessionStatus(sessionId, 'idle') - this.emitMessageRefresh(sessionId, messageId) - } - private updatePersistedProviderPermissionState( - messageId: string, - toolCallId: string, - requestId: string, - permissionType: 'read' | 'write' | 'all' | 'command', - granted: boolean, - deniedMessage = 'User denied the request.' + input: ProviderPermissionInteractionInput, + projection: ProviderPermissionProjection ): void { - const message = this.messageStore.getMessage(messageId) + const message = this.messageStore.getMessage(input.messageId) if (!message || message.role !== 'assistant') { return } const blocks = this.parseAssistantBlocks(message.content) + if (!this.applyProviderPermissionProjection(blocks, input, projection)) { + return + } + this.messageStore.updateAssistantContent(input.messageId, blocks) + } + + private updateActiveProviderPermissionState( + ownerRun: LoopRun, + input: ProviderPermissionInteractionInput, + projection: ProviderPermissionProjection + ): void { + const streamState = ownerRun.streamState as StreamState + if (!Array.isArray(streamState.blocks)) { + return + } + if (this.applyProviderPermissionProjection(streamState.blocks, input, projection)) { + streamState.dirty = true + } + } + + private applyProviderPermissionProjection( + blocks: AssistantMessageBlock[], + input: ProviderPermissionInteractionInput, + projection: ProviderPermissionProjection + ): boolean { const actionBlock = blocks.find( (block) => block.type === 'action' && block.action_type === 'tool_call_permission' && - block.tool_call?.id === toolCallId && - (block.extra?.permissionRequestId === requestId || requestId === '') + block.tool_call?.id === input.toolCallId && + (block.extra?.permissionRequestId === input.requestId || input.requestId === '') ) if (!actionBlock) { + return false + } + + if (projection.status === 'resolved') { + this.markPermissionResolved(actionBlock, projection.granted, input.permissionType) + return true + } + + actionBlock.status = 'error' + actionBlock.content = projection.message + actionBlock.extra = { + ...actionBlock.extra, + needsUserAction: false + } + this.updateToolCallResponse(blocks, input.toolCallId, projection.message, true) + return true + } + + private failProviderPermissionInteraction( + input: ProviderPermissionInteractionInput, + errorMessage: string, + instance?: DeepChatAgentInstance + ): void { + const message = this.messageStore.getMessage(input.messageId) + if (!message || message.role !== 'assistant') { return } - this.markPermissionResolved(actionBlock, granted, permissionType) - if (!granted) { - actionBlock.content = deniedMessage + const blocks = this.parseAssistantBlocks(message.content) + this.applyProviderPermissionProjection(blocks, input, { + status: 'error', + message: errorMessage + }) + const terminalBlocks = buildTerminalErrorBlocks(blocks, errorMessage) + const terminalMetadata = stampTerminalMetadata( + parseMessageMetadata(message.metadata), + 'error', + 'provider_error' + ) + this.messageStore.setMessageError( + input.messageId, + terminalBlocks, + JSON.stringify(terminalMetadata) + ) + this.emitMessageRefresh(input.sessionId, input.messageId) + publishDeepchatEvent('chat.stream.failed', { + requestId: this.resolveStreamRequestId(input.sessionId, input.messageId), + sessionId: input.sessionId, + messageId: input.messageId, + failedAt: Date.now(), + error: errorMessage + }) + this.dispatchTerminalHooks(input.sessionId, this.getDeepChatRuntimeState(input.sessionId), { + status: 'error', + stopReason: 'provider_error', + errorMessage, + usage: buildUsageFromMetadata(terminalMetadata) + }) + if (instance) { + this.removePendingProviderPermission(instance, input) + if (!instance.getActiveGeneration()) { + this.setSessionStatus(input.sessionId, 'error') + } } - this.messageStore.updateAssistantContent(messageId, blocks) + } + + private removePendingProviderPermission( + instance: DeepChatAgentInstance, + input: ProviderPermissionInteractionInput + ): void { + instance.replacePendingInteractions( + instance + .getPendingInteractions() + .filter( + (interaction) => + interaction.messageId !== input.messageId || interaction.toolCallId !== input.toolCallId + ) + ) } private clearActiveProviderPermissionsForSession(sessionId: string): void { @@ -6167,16 +6683,33 @@ export class AgentRuntimePresenter { } } - private markQuestionResolved(block: AssistantMessageBlock, answerText: string): void { + private markQuestionResolved( + block: AssistantMessageBlock, + answerText: string, + awaitsUserFollowUp = false + ): void { block.status = 'success' block.extra = { ...block.extra, needsUserAction: false, questionResolution: 'replied', + questionFollowUpPending: awaitsUserFollowUp, ...(answerText ? { answerText } : {}) } } + private hasQuestionFollowUpIntent(blocks: AssistantMessageBlock[]): boolean { + return blocks.some( + (block) => + block.type === 'action' && + block.action_type === 'question_request' && + block.status === 'success' && + block.extra?.needsUserAction === false && + block.extra?.questionResolution === 'replied' && + block.extra?.questionFollowUpPending === true + ) + } + private markPermissionResolved( block: AssistantMessageBlock, granted: boolean, @@ -6331,7 +6864,8 @@ export class AgentRuntimePresenter { private async executeDeferredToolCall( sessionId: string, messageId: string, - toolCall: NonNullable + toolCall: NonNullable, + onToolCallStarted?: () => void ): Promise { if (!this.toolExecutionPort) { return { @@ -6348,56 +6882,65 @@ export class AgentRuntimePresenter { } } - const projectDir = this.resolveProjectDir(sessionId) - const sessionState = await this.getSessionState(sessionId) - const toolDefinitions = await this.loadToolDefinitionsForSession(sessionId, projectDir) + const deferredAbortController = toolCall.id + ? this.registerDeferredToolAbortController(sessionId, toolCall.id) + : null + const deferredAbortSignal = + deferredAbortController?.signal ?? this.getAbortSignalForSession(sessionId) + let invoked = false - const toolDefinition = toolDefinitions.find((definition) => { - if (definition.function.name !== toolName) { - return false - } - if (toolCall.server_name) { - return definition.server.name === toolCall.server_name - } - return true - }) + try { + this.throwIfAbortRequested(deferredAbortSignal) + const projectDir = this.resolveProjectDir(sessionId) + const sessionState = await awaitWithAbort( + this.getSessionState(sessionId), + deferredAbortSignal + ) + const toolDefinitions = await awaitWithAbort( + this.loadToolDefinitionsForSession(sessionId, projectDir), + deferredAbortSignal + ) + this.throwIfAbortRequested(deferredAbortSignal) + + const toolDefinition = toolDefinitions.find((definition) => { + if (definition.function.name !== toolName) { + return false + } + if (toolCall.server_name) { + return definition.server.name === toolCall.server_name + } + return true + }) - if (!toolDefinition) { - const disabledAgentTools = this.getDisabledAgentTools(sessionId) - if (disabledAgentTools.includes(toolName)) { + if (!toolDefinition) { + const disabledAgentTools = this.getDisabledAgentTools(sessionId) return { - responseText: `Tool '${toolName}' is disabled for the current session.`, + responseText: disabledAgentTools.includes(toolName) + ? `Tool '${toolName}' is disabled for the current session.` + : `Tool '${toolName}' is no longer available in the current session.`, isError: true } } - return { - responseText: `Tool '${toolName}' is no longer available in the current session.`, - isError: true + const request: MCPToolCall = { + id: toolCall.id || '', + type: 'function', + function: { + name: toolName, + arguments: toolCall.params || '{}' + }, + server: toolDefinition.server, + conversationId: sessionId, + providerId: sessionState?.providerId?.trim() || undefined } - } - - const request: MCPToolCall = { - id: toolCall.id || '', - type: 'function', - function: { - name: toolName, - arguments: toolCall.params || '{}' - }, - server: toolDefinition?.server, - conversationId: sessionId, - providerId: sessionState?.providerId?.trim() || undefined - } - const deferredAbortController = toolCall.id - ? this.registerDeferredToolAbortController(sessionId, toolCall.id) - : null - const deferredAbortSignal = - deferredAbortController?.signal ?? this.getAbortSignalForSession(sessionId) - let invoked = false - try { - const extensionPolicy = await this.resolveAgentExtensionPolicy(sessionId) + const extensionPolicy = await awaitWithAbort( + this.resolveAgentExtensionPolicy(sessionId), + deferredAbortSignal + ) + this.throwIfAbortRequested(deferredAbortSignal) invoked = true + onToolCallStarted?.() const result = await this.toolExecutionPort.execute(request, { agentId: this.getSessionAgentId(sessionId) ?? 'deepchat', enabledSkillNames: extensionPolicy.enabledSkillNames ?? undefined, @@ -6419,6 +6962,7 @@ export class AgentRuntimePresenter { }, signal: deferredAbortSignal }) + this.throwIfAbortRequested(deferredAbortSignal) const rawData = result.rawData as MCPToolResponse if (rawData.requiresPermission) { return { @@ -6460,8 +7004,10 @@ export class AgentRuntimePresenter { toolName, toolArgs: toolCall.params || '{}', content: rawData.content, - cacheImage: this.cacheImage + cacheImage: this.cacheImage, + signal: deferredAbortSignal })) + this.throwIfAbortRequested(deferredAbortSignal) const normalizedContent = await this.toolResultPort.normalize({ sessionId, toolCallId: toolCall.id || '', @@ -6471,13 +7017,18 @@ export class AgentRuntimePresenter { isError: rawData.isError === true, signal: deferredAbortSignal }) + this.throwIfAbortRequested(deferredAbortSignal) const responseText = this.toolContentToText(normalizedContent) - const prepared = await this.toolResultPort.prepare({ - sessionId, - toolCallId: toolCall.id || '', - toolName, - rawContent: responseText - }) + const prepared = await awaitWithAbort( + this.toolResultPort.prepare({ + sessionId, + toolCallId: toolCall.id || '', + toolName, + rawContent: responseText + }), + deferredAbortSignal + ) + this.throwIfAbortRequested(deferredAbortSignal) if (prepared.kind === 'tool_error') { return { responseText: prepared.message, @@ -6498,6 +7049,9 @@ export class AgentRuntimePresenter { imagePreviews } } catch (error) { + if (deferredAbortSignal?.aborted) { + throw error + } const errorText = error instanceof Error ? error.message : String(error) return { responseText: `Error: ${errorText}`, @@ -7007,7 +7561,7 @@ export class AgentRuntimePresenter { block.status === 'success' && block.extra?.needsUserAction === false && block.extra?.questionResolution === 'replied' && - typeof block.extra?.answerText !== 'string' + block.extra?.questionFollowUpPending === true ) }) } @@ -7074,6 +7628,7 @@ export class AgentRuntimePresenter { this.summaryStateToCompactionState(intent.previousState), expectedInstance ) + this.throwIfAbortRequested(options?.signal) } throw error } @@ -7271,7 +7826,9 @@ export class AgentRuntimePresenter { const instance = expectedInstance if (incoming !== undefined) { const normalized = this.normalizeProjectDir(incoming) - const previous = instance.getProjectDir() + const previous = instance.hasProjectDir() + ? instance.getProjectDir() + : this.resolvePersistedSessionProjectDir(sessionId) instance.setProjectDir(normalized) if (previous !== normalized) { instance.invalidateResourceCaches() diff --git a/src/main/presenter/agentRuntimePresenter/messageStore.ts b/src/main/presenter/agentRuntimePresenter/messageStore.ts index f2f1585ac..71e4de2bb 100644 --- a/src/main/presenter/agentRuntimePresenter/messageStore.ts +++ b/src/main/presenter/agentRuntimePresenter/messageStore.ts @@ -214,9 +214,21 @@ export class DeepChatMessageStore { return messageId } - updateAssistantContent(messageId: string, blocks: AssistantMessageBlock[]): void { + updateAssistantContent( + messageId: string, + blocks: AssistantMessageBlock[], + metadata?: string + ): void { this.sqlitePresenter.deepchatAssistantBlocksTable.replaceForMessage(messageId, blocks) this.sqlitePresenter.deepchatMessagesTable.updateStatus(messageId, 'pending') + if (metadata !== undefined) { + this.updateAssistantMetadata(messageId, metadata) + } + } + + updateAssistantMetadata(messageId: string, metadata: string): void { + this.sqlitePresenter.deepchatMessagesTable.updateMetadata(messageId, metadata) + this.persistUsageStats(messageId, metadata, 'live') } updateMessageStatus(messageId: string, status: 'pending' | 'sent' | 'error'): void { diff --git a/src/main/presenter/agentRuntimePresenter/noProgressToolLoopGuard.ts b/src/main/presenter/agentRuntimePresenter/noProgressToolLoopGuard.ts new file mode 100644 index 000000000..3c4fa2780 --- /dev/null +++ b/src/main/presenter/agentRuntimePresenter/noProgressToolLoopGuard.ts @@ -0,0 +1,273 @@ +import { createHash } from 'node:crypto' +import type { AgentNoProgressToolLoopMetadata } from '@shared/types/agent-interface' +import type { ChatMessage } from '@shared/types/core/chat-message' +import type { ToolCallResult } from './types' + +export const NO_PROGRESS_TERMINAL_ERROR = + 'Agent stopped after four identical tool batches produced no progress.' + +const CORRECTION_BATCH_COUNT = 2 +const TERMINAL_BATCH_COUNT = 4 + +const NO_PROGRESS_CORRECTION = { + type: 'agent_no_progress', + repeatedBatchCount: CORRECTION_BATCH_COUNT, + instruction: + 'The same tool batch returned exactly the same results. Change strategy or finalize with the available evidence; do not repeat the same calls.' +} as const + +export interface ToolBatchProgressObservation { + repeatedBatchCount: number + correctionAppended: boolean + shouldTerminate: boolean + snapshot: AgentNoProgressToolLoopMetadata +} + +export type ResumableToolBatch = { + toolCalls: ToolCallResult[] + batchMessages: ChatMessage[] +} + +const WEAK_ACKNOWLEDGEMENTS = new Set(['ok', 'success', 'succeeded', 'done', 'true', 'completed']) +const ISO_TIMESTAMP_PATTERN = + /\b\d{4}-\d{2}-\d{2}[tT]\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:[zZ]|[+-]\d{2}:?\d{2})\b/g +const VOLATILE_UUID_LABEL_PATTERN = [ + 'request(?:[\\s_-]?id)?', + 'tool[\\s_-]?call(?:[\\s_-]?id)?', + 'trace(?:[\\s_-]?id)?', + 'run(?:[\\s_-]?id)?', + 'call(?:[\\s_-]?id)?', + 'invocation(?:[\\s_-]?id)?', + 'execution(?:[\\s_-]?id)?', + 'event(?:[\\s_-]?id)?', + 'nonce' +].join('|') +const UUID_TOKEN_PATTERN = '[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}' +const LABELED_VOLATILE_UUID_PATTERN = new RegExp( + `\\b(${VOLATILE_UUID_LABEL_PATTERN})\\b\\s*[:=#-]?\\s*(${UUID_TOKEN_PATTERN})\\b`, + 'gi' +) +const VOLATILE_RESULT_KEYS = new Set([ + 'timestamp', + 'time', + 'createdat', + 'updatedat', + 'completedat', + 'startedat', + 'finishedat', + 'requestid', + 'toolcallid', + 'traceid', + 'runid', + 'callid', + 'invocationid', + 'executionid', + 'eventid', + 'nonce' +]) + +function stableJsonStringify(value: unknown): string { + if (value === null || typeof value !== 'object') { + return JSON.stringify(value) ?? 'null' + } + if (Array.isArray(value)) { + return `[${value.map((item) => stableJsonStringify(item)).join(',')}]` + } + + return `{${Object.entries(value as Record) + .sort(([left], [right]) => left.localeCompare(right)) + .map(([key, entry]) => `${JSON.stringify(key)}:${stableJsonStringify(entry)}`) + .join(',')}}` +} + +function normalizeVolatileString(value: string): string { + return value + .replace(ISO_TIMESTAMP_PATTERN, '') + .replace(LABELED_VOLATILE_UUID_PATTERN, '$1 ') +} + +function normalizeResultValue(value: unknown, key?: string): unknown { + const normalizedKey = key?.replace(/[_-]/g, '').toLowerCase() + if (normalizedKey && VOLATILE_RESULT_KEYS.has(normalizedKey)) { + return '' + } + if (typeof value === 'string') { + return normalizeVolatileString(value) + } + if (Array.isArray(value)) { + return value.map((item) => normalizeResultValue(item)) + } + if (value && typeof value === 'object') { + return Object.fromEntries( + Object.entries(value as Record).map(([entryKey, entryValue]) => [ + entryKey, + normalizeResultValue(entryValue, entryKey) + ]) + ) + } + return value +} + +function normalizeToolResult(content: ChatMessage['content']): unknown { + if (typeof content !== 'string') { + return normalizeResultValue(content ?? null) + } + + try { + const parsed = JSON.parse(content) as unknown + return typeof parsed === 'string' + ? normalizeVolatileString(parsed) + : normalizeResultValue(parsed) + } catch { + return normalizeVolatileString(content.trim()) + } +} + +function isWeakAcknowledgement(value: unknown): boolean { + if (typeof value === 'boolean') return value + if (typeof value === 'string') { + return WEAK_ACKNOWLEDGEMENTS.has(value.trim().toLowerCase()) + } + if (!value || typeof value !== 'object' || Array.isArray(value)) return false + + const meaningfulEntries = Object.entries(value as Record).filter( + ([key]) => !VOLATILE_RESULT_KEYS.has(key.replace(/[_-]/g, '').toLowerCase()) + ) + if (meaningfulEntries.length === 0) return true + if (meaningfulEntries.length > 2) return false + + return meaningfulEntries.every(([key, entryValue]) => { + const normalizedKey = key.replace(/[_-]/g, '').toLowerCase() + return ( + (normalizedKey === 'ok' || + normalizedKey === 'success' || + normalizedKey === 'status' || + normalizedKey === 'result') && + isWeakAcknowledgement(entryValue) + ) + }) +} + +function canonicalizeToolArguments(rawArguments: string): string { + try { + return stableJsonStringify(JSON.parse(rawArguments)) + } catch { + return rawArguments.trim() + } +} + +function buildToolBatchFingerprint( + toolCalls: ToolCallResult[], + messages: ChatMessage[] +): { + fingerprint: string + evidence: AgentNoProgressToolLoopMetadata['evidence'] +} { + const calls = toolCalls.map((toolCall) => ({ + name: toolCall.name.trim(), + arguments: canonicalizeToolArguments(toolCall.arguments) + })) + const results = messages + .filter((message) => message.role === 'tool') + .map((message) => normalizeToolResult(message.content)) + const fingerprint = createHash('sha256') + .update(stableJsonStringify({ calls, results })) + .digest('hex') + .slice(0, 24) + + return { + fingerprint, + evidence: results.length > 0 && results.every(isWeakAcknowledgement) ? 'weak' : 'strong' + } +} + +function appendCorrection(messages: ChatMessage[]): boolean { + const lastToolMessageIndex = messages.findLastIndex((message) => message.role === 'tool') + if (lastToolMessageIndex < 0) { + return false + } + + const lastToolMessage = messages[lastToolMessageIndex] + const correction = JSON.stringify(NO_PROGRESS_CORRECTION) + const content = + typeof lastToolMessage.content === 'string' + ? `${lastToolMessage.content}\n\n${correction}` + : [...(lastToolMessage.content ?? []), { type: 'text' as const, text: correction }] + messages[lastToolMessageIndex] = { ...lastToolMessage, content } + return true +} + +export class NoProgressToolLoopGuard { + private previousFingerprint: string | null = null + private repeatedBatchCount = 0 + + constructor(snapshot?: AgentNoProgressToolLoopMetadata) { + if ( + snapshot && + typeof snapshot.fingerprint === 'string' && + snapshot.fingerprint.length > 0 && + Number.isInteger(snapshot.repeatedBatchCount) && + snapshot.repeatedBatchCount > 0 && + (snapshot.evidence === 'strong' || snapshot.evidence === 'weak') + ) { + this.previousFingerprint = snapshot.fingerprint + this.repeatedBatchCount = snapshot.repeatedBatchCount + } + } + + observe(toolCalls: ToolCallResult[], batchMessages: ChatMessage[]): ToolBatchProgressObservation { + const { fingerprint, evidence } = buildToolBatchFingerprint(toolCalls, batchMessages) + if (fingerprint === this.previousFingerprint) { + this.repeatedBatchCount += 1 + } else { + this.previousFingerprint = fingerprint + this.repeatedBatchCount = 1 + } + + const correctionAppended = + this.repeatedBatchCount === CORRECTION_BATCH_COUNT && appendCorrection(batchMessages) + + return { + repeatedBatchCount: this.repeatedBatchCount, + correctionAppended, + shouldTerminate: evidence === 'strong' && this.repeatedBatchCount >= TERMINAL_BATCH_COUNT, + snapshot: { + fingerprint, + repeatedBatchCount: this.repeatedBatchCount, + evidence + } + } + } +} + +export function extractLatestCompletedToolBatch( + messages: ChatMessage[] +): ResumableToolBatch | null { + const lastUserIndex = messages.findLastIndex((message) => message.role === 'user') + for (let index = messages.length - 1; index > lastUserIndex; index -= 1) { + const message = messages[index] + if (message.role !== 'assistant' || !message.tool_calls?.length) continue + + const followingMessages = messages.slice(index + 1) + if (followingMessages.some((entry) => entry.role !== 'tool')) return null + + const toolMessagesById = new Map( + followingMessages + .filter((entry) => entry.role === 'tool' && entry.tool_call_id) + .map((entry) => [entry.tool_call_id!, entry]) + ) + if (message.tool_calls.some((toolCall) => !toolMessagesById.has(toolCall.id))) return null + + return { + toolCalls: message.tool_calls.map((toolCall) => ({ + id: toolCall.id, + name: toolCall.function.name, + arguments: toolCall.function.arguments, + ...(toolCall.provider_options ? { providerOptions: toolCall.provider_options } : {}) + })), + batchMessages: message.tool_calls.map((toolCall) => toolMessagesById.get(toolCall.id)!) + } + } + + return null +} diff --git a/src/main/presenter/agentRuntimePresenter/process.ts b/src/main/presenter/agentRuntimePresenter/process.ts index 643b2b738..b0455d0b0 100644 --- a/src/main/presenter/agentRuntimePresenter/process.ts +++ b/src/main/presenter/agentRuntimePresenter/process.ts @@ -9,17 +9,21 @@ import type { ProcessResult, StreamState } from './types' -import { accumulate, finalizeTrailingPendingNarrativeBlocks } from './accumulator' +import { accumulate, commitRoundUsage, finalizeTrailingPendingNarrativeBlocks } from './accumulator' import { startEcho } from './echo' import { executeTools, finalize, finalizeError, finalizePaused, - publishPlanUpdated, - persistAbortExceptionPlanState + publishPlanUpdated } from './dispatch' import { isContextWindowErrorLike } from './contextWindowError' +import { + extractLatestCompletedToolBatch, + NoProgressToolLoopGuard, + NO_PROGRESS_TERMINAL_ERROR +} from './noProgressToolLoopGuard' import { DeepChatLoopEngine, type DeepChatLoopCommitCallbacks, @@ -37,6 +41,13 @@ type PendingPermissionPayload = NonNullable type ToolRoundBatch = { prevBlockCount: number } +class MaxProviderRoundsError extends Error { + constructor(limit: number) { + super(`Maximum agent turns exceeded (${limit}).`) + this.name = 'MaxProviderRoundsError' + } +} + function isAbortError(error: unknown): boolean { return error instanceof Error && (error.name === 'AbortError' || error.name === 'CanceledError') } @@ -58,22 +69,93 @@ function stripTrailingErrorBlock(state: StreamState, message: string): void { } } +export const MAX_TOOL_CALLS_SKIPPED_ERROR = + 'Tool call was not executed because the maximum tool-call limit was reached.' + +function stampRunAccounting(state: StreamState): void { + state.metadata.providerRounds = state.providerRoundCount + state.metadata.toolCalls = state.toolCallCount +} + +function stampRunOutcome( + state: StreamState, + outcome: 'completed' | 'paused' | 'aborted' | 'error', + stopReason: string +): void { + state.metadata.runOutcome = outcome + state.metadata.runStopReason = stopReason +} + +function toNonNegativeNumber(value: number | undefined): number | undefined { + return typeof value === 'number' && Number.isFinite(value) && value >= 0 ? value : undefined +} + +function toPositiveInteger(value: number | undefined): number | undefined { + return typeof value === 'number' && Number.isInteger(value) && value > 0 ? value : undefined +} + +function markUnexecutedToolCallsForLimit(state: StreamState): void { + const unexecutedIds = new Set(state.completedToolCalls.map((toolCall) => toolCall.id)) + for (const block of state.blocks) { + if ( + block.type !== 'tool_call' || + !block.tool_call?.id || + !unexecutedIds.has(block.tool_call.id) || + (block.status !== 'pending' && block.status !== 'loading') + ) { + continue + } + + block.status = 'error' + block.tool_call.response = MAX_TOOL_CALLS_SKIPPED_ERROR + block.extra = { + ...block.extra, + toolCallSkippedReason: 'max_tool_calls' + } + state.dirty = true + } +} + export type ProviderTerminalDecision = - | { type: 'complete' } - | { type: 'error'; error: string; source: 'context' | 'empty' } + | { type: 'complete'; stopReason: 'complete' | 'max_tokens' | 'max_tool_calls' } + | { + type: 'error' + error: string + source: 'context' | 'empty' | 'provider' + stopReason: 'context_window' | 'empty_response' | 'provider_error' + } export function resolveProviderTerminalDecision(state: StreamState): ProviderTerminalDecision { if (state.stopReason === 'error') { - const streamErrorMessage = getLatestErrorMessage(state) - if (streamErrorMessage && isContextWindowErrorLike(streamErrorMessage)) { + const streamErrorMessage = getLatestErrorMessage(state) ?? NO_MODEL_RESPONSE_ERROR + if (isContextWindowErrorLike(streamErrorMessage)) { stripTrailingErrorBlock(state, streamErrorMessage) - return { type: 'error', error: streamErrorMessage, source: 'context' } + return { + type: 'error', + error: streamErrorMessage, + source: 'context', + stopReason: 'context_window' + } + } + return { + type: 'error', + error: streamErrorMessage, + source: 'provider', + stopReason: 'provider_error' } } if (state.blocks.length === 0 && !state.latestAgentPlanSnapshot) { - return { type: 'error', error: NO_MODEL_RESPONSE_ERROR, source: 'empty' } + return { + type: 'error', + error: NO_MODEL_RESPONSE_ERROR, + source: 'empty', + stopReason: 'empty_response' + } + } + if (state.stopReason === 'max_tokens' || state.stopReason === 'max_tool_calls') { + return { type: 'complete', stopReason: state.stopReason } } - return { type: 'complete' } + return { type: 'complete', stopReason: 'complete' } } function parseAssistantBlocks(rawContent: string): AssistantMessageBlock[] { @@ -305,6 +387,21 @@ function replaceLeadingSystemMessage(messages: ChatMessage[], systemPrompt: stri messages.unshift({ role: 'system', content: systemPrompt }) } +function commitCorrectedToolMessage( + conversationMessages: ChatMessage[], + batchMessages: ChatMessage[] +): void { + const correctedMessage = batchMessages.findLast((message) => message.role === 'tool') + if (!correctedMessage?.tool_call_id) return + + const messageIndex = conversationMessages.findLastIndex( + (message) => message.role === 'tool' && message.tool_call_id === correctedMessage.tool_call_id + ) + if (messageIndex >= 0) { + conversationMessages[messageIndex] = correctedMessage + } +} + export function markStreamingProviderPermissionResolved( block: AssistantMessageBlock, granted: boolean, @@ -329,10 +426,13 @@ function settleLoopOutcome( run: ProcessParams['run'], outputSink: OutputSink ): ProcessResult { + stampRunAccounting(state) if (outcome.type === 'thrown') { + commitRoundUsage(state) if (io.abortSignal.aborted || isAbortError(outcome.error)) { logger.info(`[ProcessStream] aborted via exception after ${eventCount} events`) - persistAbortExceptionPlanState(state, io) + stampRunOutcome(state, 'aborted', 'user_stop') + finalizeUserCanceledErrorIfNeeded(state, io) return { status: 'aborted', stopReason: 'user_stop', @@ -341,7 +441,30 @@ function settleLoopOutcome( } } + if (outcome.error instanceof MaxProviderRoundsError) { + logger.info(`[ProcessStream] ${outcome.error.message}`) + stampRunOutcome(state, 'error', 'max_turns') + outputSink.fail({ + runId: run.runId, + sessionId: run.sessionId, + messageId: run.messageId, + error: outcome.error + }) + return { + status: 'error', + terminalError: outcome.error.message, + stopReason: 'max_turns', + errorMessage: outcome.error.message, + usage: buildUsageSnapshot(state) + } + } + console.error(`[ProcessStream] exception after ${eventCount} events:`, outcome.error) + const errorMessage = + outcome.error instanceof Error ? outcome.error.message : String(outcome.error) + const contextWindowError = isContextWindowErrorLike(outcome.error) + const stopReason = contextWindowError ? 'context_window' : 'provider_error' + stampRunOutcome(state, 'error', stopReason) outputSink.fail({ runId: run.runId, sessionId: run.sessionId, @@ -350,8 +473,9 @@ function settleLoopOutcome( }) return { status: 'error', - stopReason: 'error', - errorMessage: outcome.error instanceof Error ? outcome.error.message : String(outcome.error), + terminalError: errorMessage, + stopReason, + errorMessage, usage: buildUsageSnapshot(state) } } @@ -359,10 +483,17 @@ function settleLoopOutcome( if (outcome.type === 'halted') { const result = outcome.result if (result.status === 'paused') { + stampRunOutcome(state, 'paused', result.stopReason ?? 'interaction') finalizePaused(state, io) } else if (result.status === 'aborted') { + stampRunOutcome(state, 'aborted', result.stopReason ?? 'user_stop') finalizeUserCanceledErrorIfNeeded(state, io) } else if (result.status === 'error') { + stampRunOutcome( + state, + 'error', + result.stopReason ?? (result.terminalError ? 'tool_error' : 'provider_error') + ) outputSink.fail({ runId: run.runId, sessionId: run.sessionId, @@ -370,6 +501,7 @@ function settleLoopOutcome( error: result.terminalError ?? result.errorMessage ?? 'Unknown error' }) } else { + stampRunOutcome(state, 'completed', result.stopReason ?? 'complete') outputSink.complete({ runId: run.runId, sessionId: run.sessionId, @@ -384,6 +516,7 @@ function settleLoopOutcome( if (outcome.type === 'max_provider_rounds') { const errorMessage = `Maximum agent turns exceeded (${outcome.limit}).` logger.info(`[ProcessStream] ${errorMessage}`) + stampRunOutcome(state, 'error', 'max_turns') outputSink.fail({ runId: run.runId, sessionId: run.sessionId, @@ -403,10 +536,13 @@ function settleLoopOutcome( logger.info( `[ProcessStream] max tool calls reached (${outcome.attemptedToolCount} > ${outcome.limit}), stopping` ) + state.stopReason = 'max_tool_calls' state.planTerminalReason = 'max_steps' + markUnexecutedToolCallsForLimit(state) } if (io.abortSignal.aborted) { + stampRunOutcome(state, 'aborted', 'user_stop') finalizeUserCanceledErrorIfNeeded(state, io) return { status: 'aborted', @@ -417,27 +553,23 @@ function settleLoopOutcome( } const terminalDecision = resolveProviderTerminalDecision(state) if (terminalDecision.type === 'error') { + stampRunOutcome(state, 'error', terminalDecision.stopReason) outputSink.fail({ runId: run.runId, sessionId: run.sessionId, messageId: run.messageId, error: terminalDecision.error }) - if (terminalDecision.source === 'context') { - return { - status: 'error', - terminalError: terminalDecision.error - } - } return { status: 'error', terminalError: terminalDecision.error, - stopReason: 'error', + stopReason: terminalDecision.stopReason, errorMessage: terminalDecision.error, usage: buildUsageSnapshot(state) } } + stampRunOutcome(state, 'completed', terminalDecision.stopReason) outputSink.complete({ runId: run.runId, sessionId: run.sessionId, @@ -447,7 +579,7 @@ function settleLoopOutcome( }) return { status: 'completed', - stopReason: 'complete', + stopReason: terminalDecision.stopReason, usage: buildUsageSnapshot(state) } } @@ -486,16 +618,51 @@ export async function processStream(params: ProcessParams): Promise 0) { state.blocks = JSON.parse(JSON.stringify(initialBlocks)) as typeof state.blocks } + state.metadata.runId = run.runId const echo = startEcho(state, io) const conversationMessages = run.messages params.onConversationMessagesChange?.(conversationMessages) let currentTools = run.resources.toolDefinitions let firstProviderRoundReady = false + const noProgressToolLoopGuard = new NoProgressToolLoopGuard(state.metadata.noProgressToolLoop) const outputSink: OutputSink = { update: () => echo.flush(), complete: () => finalize(state, io), @@ -556,19 +723,76 @@ export async function processStream(params: ProcessParams): Promise( run, { - maxProviderRounds: params.maxProviderRounds, + maxProviderRounds, + initialExecutedToolCount: state.toolCallCount, consumeProviderRound: async () => { const prevBlockCount = state.blocks.length + const assertProviderRequestAvailable = (): void => { + if (maxProviderRounds !== undefined && state.providerRoundCount >= maxProviderRounds) { + throw new MaxProviderRoundsError(maxProviderRounds) + } + } + const markProviderRequestStarted = (): void => { + assertProviderRequestAvailable() + state.providerRoundCount += 1 + state.metadata.providerRounds = state.providerRoundCount + } + if (params.coreStreamReportsProviderStart !== true) { + markProviderRequestStarted() + } else { + assertProviderRequestAvailable() + } const stream = coreStream( conversationMessages, modelId, modelConfig, temperature, maxTokens, - currentTools + currentTools, + params.coreStreamReportsProviderStart === true ? markProviderRequestStarted : undefined, + params.coreStreamReportsProviderStart === true + ? assertProviderRequestAvailable + : undefined ) // Reset per-iteration accumulator state @@ -580,6 +804,7 @@ export async function processStream(params: ProcessParams): Promise { // A completed tool call implies that the tool presenter and definitions were available. - const executed = await executeTools( - state, - conversationMessages, - batch.prevBlockCount, - currentTools, - toolExecution!, - modelId, - interleavedReasoning, - io, - permissionMode, - toolResults, - providerId === 'acp' - ? Number.MAX_SAFE_INTEGER - : modelConfig.contextLength > 0 - ? modelConfig.contextLength - : UNKNOWN_CONTEXT_LIMIT, - maxTokens, - echo, - { - notificationObserver, - controls, - diagnostics - }, - providerId - ) + const completedToolBatch = state.completedToolCalls.map((toolCall) => ({ ...toolCall })) + const toolBatchMessageStart = conversationMessages.length + let startedToolCallCount = 0 + let executed: Awaited> + try { + executed = await executeTools( + state, + conversationMessages, + batch.prevBlockCount, + currentTools, + toolExecution!, + modelId, + interleavedReasoning, + io, + permissionMode, + toolResults, + providerId === 'acp' + ? Number.MAX_SAFE_INTEGER + : modelConfig.contextLength > 0 + ? modelConfig.contextLength + : UNKNOWN_CONTEXT_LIMIT, + maxTokens, + echo, + { + notificationObserver, + controls, + diagnostics, + onToolCallStarted: () => { + startedToolCallCount += 1 + } + }, + providerId + ) + } finally { + state.toolCallCount += startedToolCallCount + state.metadata.toolCalls = state.toolCallCount + } if (executed.type === 'completed' && executed.terminalError) { return { @@ -686,7 +925,7 @@ export async function processStream(params: ProcessParams): Promise, + permissionMode?: PermissionMode + ): void { + const update = (): void => { + this.sqlitePresenter.deepchatSessionsTable.updateSessionModel(id, providerId, modelId) + if (permissionMode !== undefined) { + this.sqlitePresenter.deepchatSessionsTable.updatePermissionMode(id, permissionMode) + } + this.sqlitePresenter.deepchatSessionsTable.updateGenerationSettings(id, generationSettings) + } + + const db = this.sqlitePresenter.getDatabase?.() + if (db) { + db.transaction(update)() + return + } + update() + } + getSummaryState(id: string): SessionSummaryState { const tapeTable = this.sqlitePresenter.deepchatTapeEntriesTable const tapeState = summaryStateFromTapeAnchor( diff --git a/src/main/presenter/agentRuntimePresenter/tapeService.ts b/src/main/presenter/agentRuntimePresenter/tapeService.ts index d7a88034d..6fc09f009 100644 --- a/src/main/presenter/agentRuntimePresenter/tapeService.ts +++ b/src/main/presenter/agentRuntimePresenter/tapeService.ts @@ -1491,6 +1491,15 @@ export class DeepChatTapeService implements Pick }) } + handoffResult( + sessionId: string, + name: string, + state: Record = {}, + meta: Record = {} + ): TapeAnchorResult { + return this.toAnchorResult(this.handoff(sessionId, name, state, meta)) + } + createFork(parentSessionId: string, forkId: string = nanoid()): TapeForkHandle { const table = this.table if (!table) { diff --git a/src/main/presenter/agentRuntimePresenter/types.ts b/src/main/presenter/agentRuntimePresenter/types.ts index 065a4db65..c918341c4 100644 --- a/src/main/presenter/agentRuntimePresenter/types.ts +++ b/src/main/presenter/agentRuntimePresenter/types.ts @@ -55,9 +55,18 @@ export interface StreamState { } > completedToolCalls: ToolCallResult[] - stopReason: 'complete' | 'tool_use' | 'error' | 'abort' | 'max_tokens' + stopReason: 'complete' | 'tool_use' | 'error' | 'abort' | 'max_tokens' | 'max_tool_calls' latestAgentPlanSnapshot?: AgentPlanSnapshot planTerminalReason?: AgentPlanTerminalReason + roundUsage: { + inputTokens: number + outputTokens: number + totalTokens: number + cachedInputTokens?: number + cacheWriteInputTokens?: number + } | null + providerRoundCount: number + toolCallCount: number dirty: boolean } @@ -111,6 +120,7 @@ export interface ToolDispatchCollaborators { notificationObserver?: DeepChatLoopNotificationObserver controls?: ProcessControlCollaborators diagnostics?: ProcessInternalDiagnostics + onToolCallStarted?: (toolCallId: string) => void } export interface ToolPermissionReviewRequest { @@ -201,8 +211,11 @@ export interface ProcessParams { modelConfig: ModelConfig, temperature: number, maxTokens: number, - tools: MCPToolDefinition[] + tools: MCPToolDefinition[], + onProviderRequestStart?: () => void, + assertProviderRequestAvailable?: () => void ) => AsyncGenerator + coreStreamReportsProviderStart?: boolean providerId: string modelId: string modelConfig: ModelConfig @@ -211,6 +224,7 @@ export interface ProcessParams { interleavedReasoning: InterleavedReasoningConfig permissionMode: PermissionMode initialBlocks?: AssistantMessageBlock[] + initialAccounting?: MessageMetadata onFirstProviderRoundReady?: () => void onConversationMessagesChange?: (messages: ChatMessage[]) => void shouldYieldForPendingInput?: () => boolean @@ -230,6 +244,9 @@ export function createState(): StreamState { pendingToolCalls: new Map(), completedToolCalls: [], stopReason: 'complete', + roundUsage: null, + providerRoundCount: 0, + toolCallCount: 0, dirty: false } } diff --git a/src/main/presenter/llmProviderPresenter/index.ts b/src/main/presenter/llmProviderPresenter/index.ts index d8f6f7117..b649b35d7 100644 --- a/src/main/presenter/llmProviderPresenter/index.ts +++ b/src/main/presenter/llmProviderPresenter/index.ts @@ -73,12 +73,20 @@ const createAbortPromise = ( } let abortHandler: (() => void) | null = null + let abortHandled = false const promise = new Promise((_, reject) => { abortHandler = () => { - onAbort?.() + if (abortHandled) return + abortHandled = true + try { + onAbort?.() + } catch { + // Abort cleanup is best effort; the caller must still receive the cancellation error. + } reject(createAbortError()) } signal.addEventListener('abort', abortHandler, { once: true }) + if (signal.aborted) abortHandler() }) return { @@ -91,6 +99,15 @@ const createAbortPromise = ( } } +const closeAsyncIterator = (stream: AsyncIterator): void => { + try { + const closing = stream.return?.() + if (closing) void closing.catch(() => undefined) + } catch { + // Iterator teardown is best effort and must not replace the cancellation outcome. + } +} + const AUDIO_TRANSCRIPTION_PROMPT = [ 'Transcribe the provided audio.', 'Return only the transcription text of the audio content.', @@ -408,17 +425,11 @@ export class LLMProviderPresenter } const completionPromise = provider.completions(messages, modelId, temperature, maxTokens) - const abortPromise = - signal && - new Promise((_, reject) => { - const onAbort = () => reject(createAbortError()) - signal.addEventListener('abort', onAbort, { once: true }) - completionPromise.finally(() => signal.removeEventListener('abort', onAbort)) - }) + const abort = createAbortPromise(signal) try { - const llmResponse = await (abortPromise - ? Promise.race([completionPromise, abortPromise]) + const llmResponse = await (abort.promise + ? Promise.race([completionPromise, abort.promise]) : completionPromise) response = llmResponse.content @@ -435,6 +446,8 @@ export class LLMProviderPresenter } return '' + } finally { + abort.cleanup() } } @@ -554,7 +567,7 @@ export class LLMProviderPresenter ) const images: StandaloneImageGenerationResult['images'] = [] const abort = createAbortPromise(signal, () => { - void stream.return?.(undefined as never) + closeAsyncIterator(stream) }) const collect = async () => { @@ -634,7 +647,7 @@ export class LLMProviderPresenter ) const videos: StandaloneVideoGenerationResult['videos'] = [] const abort = createAbortPromise(signal, () => { - void stream.return?.(undefined as never) + closeAsyncIterator(stream) }) const collect = async () => { diff --git a/src/main/presenter/mcpPresenter/index.ts b/src/main/presenter/mcpPresenter/index.ts index 0fee2eb11..635b185b7 100644 --- a/src/main/presenter/mcpPresenter/index.ts +++ b/src/main/presenter/mcpPresenter/index.ts @@ -767,15 +767,22 @@ export class McpPresenter implements IMCPPresenter { async callTool( request: MCPToolCall, - options?: { agentId?: string; enabledServerIds?: string[] } + options?: { + signal?: AbortSignal + agentId?: string + enabledServerIds?: string[] + } ): Promise<{ content: string; rawData: MCPToolResponse }> { const toolCallResult = await this.toolManager.callTool(request, options) + options?.signal?.throwIfAborted() const imagePreviews = await extractToolCallImagePreviews({ toolName: request.function.name, toolArgs: request.function.arguments, content: toolCallResult.content, - cacheImage: this.cacheImage + cacheImage: this.cacheImage, + signal: options?.signal }) + options?.signal?.throwIfAborted() // Format tool call results into strings that are easy for large models to parse let formattedContent = '' @@ -832,7 +839,11 @@ export class McpPresenter implements IMCPPresenter { */ async preCheckToolPermission( request: MCPToolCall, - options?: { agentId?: string; enabledServerIds?: string[] } + options?: { + signal?: AbortSignal + agentId?: string + enabledServerIds?: string[] + } ): Promise<{ needsPermission: true toolName: string diff --git a/src/main/presenter/mcpPresenter/mcpClient.ts b/src/main/presenter/mcpPresenter/mcpClient.ts index 8cfd6a34b..e0f21250f 100644 --- a/src/main/presenter/mcpPresenter/mcpClient.ts +++ b/src/main/presenter/mcpPresenter/mcpClient.ts @@ -27,6 +27,7 @@ import { getInMemoryServer } from './inMemoryServers/builder' import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js' import { RuntimeHelper } from '@/lib/runtimeHelper' import { terminateProcessTree } from '@/agent/shared/process/processTree' +import { awaitWithAbort } from '@/lib/awaitWithAbort' import type { McpOAuthManager } from './mcpOAuthManager' import { PromptListEntry, @@ -168,6 +169,9 @@ function isUnsupportedCapabilityError(error: unknown): boolean { } // MCP client class +const isAbortError = (error: unknown): boolean => + error instanceof Error && (error.name === 'AbortError' || error.name === 'CanceledError') + export class McpClient { private client: Client | null = null private transport: Transport | null = null @@ -309,11 +313,12 @@ export class McpClient { return this.waitForConnectSoftTimeout(connectPromise, attempt, options.phase) } - private async ensureConnectedForRequest(): Promise { + private async ensureConnectedForRequest(signal?: AbortSignal): Promise { if (!this.isConnected) { - await this.connect({ phase: 'manual', waitForConnection: true }) + await awaitWithAbort(this.connect({ phase: 'manual', waitForConnection: true }), signal) } + signal?.throwIfAborted() if (!this.isConnected || !this.client) { throw new Error(`MCP client ${this.serverName} is not connected`) } @@ -836,79 +841,77 @@ export class McpClient { const decisionPromise = presenter.mcpPresenter.handleSamplingRequest(payload) const signal = extra?.signal as AbortSignal | undefined + const decisionWait = awaitWithAbort(decisionPromise, signal) + let abortListener: (() => void) | undefined - let decision: McpSamplingDecision if (signal) { - decision = await new Promise((resolve, reject) => { - const onAbort = () => { - signal.removeEventListener('abort', onAbort) - void presenter.mcpPresenter - .cancelSamplingRequest(payload.requestId, 'cancelled by server') - .catch((error) => { - console.warn(`[MCP] Failed to cancel sampling request ${payload.requestId}:`, error) - }) - reject(new McpError(ErrorCode.RequestTimeout, 'Sampling request cancelled')) - } - - if (signal.aborted) { - onAbort() - return - } - - signal.addEventListener('abort', onAbort, { once: true }) - decisionPromise - .then((value) => { - signal.removeEventListener('abort', onAbort) - resolve(value) - }) + abortListener = () => { + void presenter.mcpPresenter + .cancelSamplingRequest(payload.requestId, 'cancelled by server') .catch((error) => { - signal.removeEventListener('abort', onAbort) - reject(error) + console.warn(`[MCP] Failed to cancel sampling request ${payload.requestId}:`, error) }) - }) - } else { - decision = await decisionPromise + } + signal.addEventListener('abort', abortListener, { once: true }) + if (signal.aborted) abortListener() } - if (!decision.approved) { - throw new McpError(ErrorCode.InvalidRequest, 'User rejected sampling request') - } + try { + let decision: McpSamplingDecision + try { + decision = await decisionWait + } catch (error) { + if (signal?.aborted || isAbortError(error)) { + throw new McpError(ErrorCode.RequestTimeout, 'Sampling request cancelled') + } + throw error + } - if (!decision.providerId || !decision.modelId) { - throw new McpError(ErrorCode.InvalidParams, 'No model selected for sampling request') - } + if (!decision.approved) { + throw new McpError(ErrorCode.InvalidRequest, 'User rejected sampling request') + } - let assistantText = '' - try { - assistantText = await presenter.llmproviderPresenter.generateCompletionStandalone( - decision.providerId, - chatMessages, - decision.modelId, - undefined, - params.maxTokens - ) - } catch (error) { - console.error(`[MCP] Sampling request failed for server ${this.serverName}:`, error) - throw new McpError( - ErrorCode.InternalError, - error instanceof Error ? error.message : 'Sampling request failed' - ) - } + if (!decision.providerId || !decision.modelId) { + throw new McpError(ErrorCode.InvalidParams, 'No model selected for sampling request') + } - const modelName = - this.resolveModelDisplayName(decision.providerId, decision.modelId) ?? decision.modelId + let assistantText = '' + try { + assistantText = await presenter.llmproviderPresenter.generateCompletionStandalone( + decision.providerId, + chatMessages, + decision.modelId, + undefined, + params.maxTokens, + { signal, swallowErrors: false } + ) + signal?.throwIfAborted() + } catch (error) { + if (signal?.aborted || isAbortError(error)) throw error + console.error(`[MCP] Sampling request failed for server ${this.serverName}:`, error) + throw new McpError( + ErrorCode.InternalError, + error instanceof Error ? error.message : 'Sampling request failed' + ) + } - const result: CreateMessageResult = { - role: 'assistant', - model: modelName, - stopReason: 'endTurn', - content: { - type: 'text', - text: assistantText ?? '' + const modelName = + this.resolveModelDisplayName(decision.providerId, decision.modelId) ?? decision.modelId + + const result: CreateMessageResult = { + role: 'assistant', + model: modelName, + stopReason: 'endTurn', + content: { + type: 'text', + text: assistantText ?? '' + } } - } - return result + return result + } finally { + if (signal && abortListener) signal.removeEventListener('abort', abortListener) + } } private resolveSamplingRequestId(extra: RequestHandlerContext): string { @@ -1204,19 +1207,31 @@ export class McpClient { } // 调用 MCP 工具 - async callTool(toolName: string, args: Record): Promise { + async callTool( + toolName: string, + args: Record, + options?: { signal?: AbortSignal } + ): Promise { try { - await this.ensureConnectedForRequest() + options?.signal?.throwIfAborted() + await this.ensureConnectedForRequest(options?.signal) + options?.signal?.throwIfAborted() if (!this.client) { throw new Error(`MCP client ${this.serverName} not initialized`) } // 调用工具 - const result = (await this.client.callTool({ + const request = { name: toolName, arguments: args - })) as ToolCallResult + } + const result = ( + options?.signal + ? await this.client.callTool(request, undefined, { signal: options.signal }) + : await this.client.callTool(request) + ) as ToolCallResult + options?.signal?.throwIfAborted() // 成功调用后重置重启标志 this.hasRestarted = false @@ -1234,8 +1249,13 @@ export class McpClient { } return result } catch (error) { + if (options?.signal?.aborted || isAbortError(error)) { + throw error + } + // 检查并处理session错误 - await this.checkAndHandleSessionError(error) + await awaitWithAbort(this.checkAndHandleSessionError(error), options?.signal) + options?.signal?.throwIfAborted() console.error(`Failed to call MCP tool ${toolName}:`, error) // 调用失败，清空工具缓存 diff --git a/src/main/presenter/mcpPresenter/toolManager.ts b/src/main/presenter/mcpPresenter/toolManager.ts index f6e25e494..4aa87b565 100644 --- a/src/main/presenter/mcpPresenter/toolManager.ts +++ b/src/main/presenter/mcpPresenter/toolManager.ts @@ -18,9 +18,13 @@ import { getErrorMessageLabels } from '@shared/i18n' import { presenter } from '@/presenter' import { getPluginToolPolicy } from '@/presenter/pluginPresenter/toolPolicyStore' import { publishDeepchatEvent } from '@/routes/publishDeepchatEvent' +import { awaitWithAbort } from '@/lib/awaitWithAbort' const CUA_PLUGIN_ID = 'com.deepchat.plugins.cua' +const isAbortError = (error: unknown): boolean => + error instanceof Error && (error.name === 'AbortError' || error.name === 'CanceledError') + type McpToolAccessContext = { enabledTools?: string[] enabledServerIds?: string[] @@ -28,6 +32,11 @@ type McpToolAccessContext = { conversationId?: string } +type ActiveToolDefinitionsRefresh = { + completion: Promise + settle: () => void +} + const normalizeStringList = (items?: string[]): string[] | undefined => { if (!Array.isArray(items)) { return undefined @@ -55,6 +64,8 @@ export class ToolManager { private cachedToolDefinitions: MCPToolDefinition[] | null = null private toolNameToTargetMap: Map | null = null + private toolDefinitionsCacheGeneration = 0 + private activeToolDefinitionsRefresh: ActiveToolDefinitionsRefresh | null = null // Session-scoped permission cache: conversationId -> Set of "serverName:permissionType" private sessionPermissions = new Map>() @@ -67,12 +78,18 @@ export class ToolManager { private handleServerListUpdate = (): void => { console.info('MCP client list updated, clearing tool definitions cache and target map.') + this.toolDefinitionsCacheGeneration += 1 + this.activeToolDefinitionsRefresh?.settle() + this.activeToolDefinitionsRefresh = null this.cachedToolDefinitions = null this.toolNameToTargetMap = null } private handleConfigChange = (): void => { console.info('MCP configuration changed, clearing cached data.') + this.toolDefinitionsCacheGeneration += 1 + this.activeToolDefinitionsRefresh?.settle() + this.activeToolDefinitionsRefresh = null this.cachedToolDefinitions = null this.toolNameToTargetMap = null } @@ -100,172 +117,214 @@ export class ToolManager { } // Get all tool definitions public async getAllToolDefinitions( - access?: string[] | McpToolAccessContext + access?: string[] | McpToolAccessContext, + options?: { signal?: AbortSignal } ): Promise { + options?.signal?.throwIfAborted() const context = normalizeToolAccessContext(access) - if (this.cachedToolDefinitions !== null && this.cachedToolDefinitions.length > 0) { + if (this.cachedToolDefinitions !== null) { return this.filterToolDefinitionsByContext(this.cachedToolDefinitions, context) } - console.info('Fetching/refreshing tool definitions and target map...') - const clients = await this.serverManager.getRunningClients() - const results: MCPToolDefinition[] = [] - // Initialize/clear the map before processing - if (this.toolNameToTargetMap) { - this.toolNameToTargetMap.clear() // Clear existing map - } else { - this.toolNameToTargetMap = new Map() // Initialize if null + const activeRefresh = this.activeToolDefinitionsRefresh + if (activeRefresh) { + await awaitWithAbort(activeRefresh.completion, options?.signal) + return await this.getAllToolDefinitions(access, options) } - if (!clients || clients.length === 0) { - console.warn('No running MCP clients found.') - this.cachedToolDefinitions = [] - // Map is already cleared or initialized as empty - return this.cachedToolDefinitions + let resolveRefresh = () => {} + const refreshCompletion = new Promise((resolve) => { + resolveRefresh = resolve + }) + const refresh: ActiveToolDefinitionsRefresh = { + completion: refreshCompletion, + settle: () => resolveRefresh() } + this.activeToolDefinitionsRefresh = refresh + + try { + const refreshGeneration = this.toolDefinitionsCacheGeneration + console.info('Fetching/refreshing tool definitions and target map...') + const clients = await awaitWithAbort(this.serverManager.getRunningClients(), options?.signal) + const results: MCPToolDefinition[] = [] + const nextToolNameToTargetMap = new Map() + + if (!clients || clients.length === 0) { + console.warn('No running MCP clients found.') + options?.signal?.throwIfAborted() + if (refreshGeneration !== this.toolDefinitionsCacheGeneration) { + if (this.activeToolDefinitionsRefresh === refresh) { + this.activeToolDefinitionsRefresh = null + } + return await this.getAllToolDefinitions(access, options) + } + this.cachedToolDefinitions = [] + this.toolNameToTargetMap = nextToolNameToTargetMap + return this.cachedToolDefinitions + } - const toolNameToServerMap: Map = new Map() - const toolsToRename: Map> = new Map() + const toolNameToServerMap: Map = new Map() + const toolsToRename: Map> = new Map() - // Pass 1: Detect conflicts - for (const client of clients) { - try { - const clientTools = await client.listTools() - this.serverManager.clearServerLastError(client.serverName) - if (!clientTools) continue - - const currentServerRenames: Set = toolsToRename.get(client.serverName) || new Set() - - for (const tool of clientTools) { - if (toolNameToServerMap.has(tool.name)) { - const originalServerName = toolNameToServerMap.get(tool.name)! - if (originalServerName !== client.serverName) { - console.warn( - `Conflict detected for tool '${tool.name}' between server '${originalServerName}' and '${client.serverName}'. Marking for rename.` - ) - // Mark original tool for rename - const originalServerRenames = toolsToRename.get(originalServerName) || new Set() - originalServerRenames.add(tool.name) - toolsToRename.set(originalServerName, originalServerRenames) - // Mark current tool for rename - currentServerRenames.add(tool.name) + // Pass 1: Detect conflicts + for (const client of clients) { + try { + const clientTools = await awaitWithAbort(client.listTools(), options?.signal) + this.serverManager.clearServerLastError(client.serverName) + if (!clientTools) continue + + const currentServerRenames: Set = + toolsToRename.get(client.serverName) || new Set() + + for (const tool of clientTools) { + if (toolNameToServerMap.has(tool.name)) { + const originalServerName = toolNameToServerMap.get(tool.name)! + if (originalServerName !== client.serverName) { + console.warn( + `Conflict detected for tool '${tool.name}' between server '${originalServerName}' and '${client.serverName}'. Marking for rename.` + ) + // Mark original tool for rename + const originalServerRenames = toolsToRename.get(originalServerName) || new Set() + originalServerRenames.add(tool.name) + toolsToRename.set(originalServerName, originalServerRenames) + // Mark current tool for rename + currentServerRenames.add(tool.name) + } + } else { + toolNameToServerMap.set(tool.name, client.serverName) } - } else { - toolNameToServerMap.set(tool.name, client.serverName) } + if (currentServerRenames.size > 0) { + toolsToRename.set(client.serverName, currentServerRenames) + } + } catch (error: unknown) { + if (options?.signal?.aborted) throw error + // Log error and notify, but continue conflict detection with other clients + const errorMessage = error instanceof Error ? error.message : String(error) + const serverName = client.serverName || 'Unknown server' + console.error( + `Pass 1 Error: Failed to get tool list from server '${serverName}':`, + errorMessage + ) + this.serverManager.setServerLastError(serverName, errorMessage) + if (!this.isPluginOwnedClient(client)) { + // Send notification for normal MCP servers. Plugin-owned MCP errors are shown in + // plugin status surfaces instead of global toasts. + const locale = this.configPresenter.getLanguage?.() || 'zh-CN' + const errorMessages = getErrorMessageLabels(locale) + const formattedMessage = + errorMessages.getMcpToolListErrorMessage + ?.replace('{serverName}', serverName) + .replace('{errorMessage}', errorMessage) || + `Failed to get tool list from server '${serverName}': ${errorMessage}` + publishDeepchatEvent('notification.error', { + title: errorMessages.getMcpToolListErrorTitle || 'Failed to get tool definitions', + message: formattedMessage, + id: `mcp-error-pass1-${serverName}-${Date.now()}`, + type: 'error' + }) + } + continue // Continue to next client } - if (currentServerRenames.size > 0) { - toolsToRename.set(client.serverName, currentServerRenames) - } - } catch (error: unknown) { - // Log error and notify, but continue conflict detection with other clients - const errorMessage = error instanceof Error ? error.message : String(error) - const serverName = client.serverName || 'Unknown server' - console.error( - `Pass 1 Error: Failed to get tool list from server '${serverName}':`, - errorMessage - ) - this.serverManager.setServerLastError(serverName, errorMessage) - if (!this.isPluginOwnedClient(client)) { - // Send notification for normal MCP servers. Plugin-owned MCP errors are shown in - // plugin status surfaces instead of global toasts. - const locale = this.configPresenter.getLanguage?.() || 'zh-CN' - const errorMessages = getErrorMessageLabels(locale) - const formattedMessage = - errorMessages.getMcpToolListErrorMessage - ?.replace('{serverName}', serverName) - .replace('{errorMessage}', errorMessage) || - `Failed to get tool list from server '${serverName}': ${errorMessage}` - publishDeepchatEvent('notification.error', { - title: errorMessages.getMcpToolListErrorTitle || 'Failed to get tool definitions', - message: formattedMessage, - id: `mcp-error-pass1-${serverName}-${Date.now()}`, - type: 'error' - }) - } - continue // Continue to next client } - } - - // Pass 2: Build results with renaming AND populate the target map - for (const client of clients) { - try { - const clientTools = await client.listTools() - this.serverManager.clearServerLastError(client.serverName) - if (!clientTools) continue - const renamesForThisServer = toolsToRename.get(client.serverName) || new Set() + // Pass 2: Build results with renaming AND populate the target map + for (const client of clients) { + try { + const clientTools = await awaitWithAbort(client.listTools(), options?.signal) + this.serverManager.clearServerLastError(client.serverName) + if (!clientTools) continue - for (const tool of clientTools) { - let finalName = tool.name - let finalDescription = tool.description - const originalName = tool.name + const renamesForThisServer = toolsToRename.get(client.serverName) || new Set() - if (renamesForThisServer.has(originalName)) { - finalName = `${client.serverName}_${originalName}` - finalDescription = `[${client.serverName}] ${tool.description}` - } + for (const tool of clientTools) { + let finalName = tool.name + let finalDescription = tool.description + const originalName = tool.name - // Validate the final name against the allowed pattern - const namePattern = /^[a-zA-Z0-9_-]+$/ - if (!namePattern.test(finalName)) { - console.error( - `Generated tool name '${finalName}' is invalid. Skipping tool '${originalName}' from server '${client.serverName}'. Please ensure the tool name matches the allowed pattern: /^[a-zA-Z0-9_-]+$/` - ) - continue // Skip adding this tool - } + if (renamesForThisServer.has(originalName)) { + finalName = `${client.serverName}_${originalName}` + finalDescription = `[${client.serverName}] ${tool.description}` + } - const properties = tool.inputSchema.properties || {} - const toolProperties = { ...properties } - for (const key in toolProperties) { - if (!toolProperties[key].description) { - toolProperties[key].description = 'Params of ' + key + // Validate the final name against the allowed pattern + const namePattern = /^[a-zA-Z0-9_-]+$/ + if (!namePattern.test(finalName)) { + console.error( + `Generated tool name '${finalName}' is invalid. Skipping tool '${originalName}' from server '${client.serverName}'. Please ensure the tool name matches the allowed pattern: /^[a-zA-Z0-9_-]+$/` + ) + continue // Skip adding this tool } - } - results.push({ - type: 'function', - function: { - name: finalName, - description: finalDescription, - parameters: { - type: 'object', - properties: toolProperties, - required: Array.isArray(tool.inputSchema.required) ? tool.inputSchema.required : [] + const properties = tool.inputSchema.properties || {} + const toolProperties = { ...properties } + for (const key in toolProperties) { + if (!toolProperties[key].description) { + toolProperties[key].description = 'Params of ' + key } - }, - server: { - name: client.serverName, - icons: client.serverConfig.icons as string, - description: client.serverConfig.descriptions as string } - }) - // Populate the target map - if (this.toolNameToTargetMap) { - this.toolNameToTargetMap.set(finalName, { client: client, originalName: originalName }) + results.push({ + type: 'function', + function: { + name: finalName, + description: finalDescription, + parameters: { + type: 'object', + properties: toolProperties, + required: Array.isArray(tool.inputSchema.required) + ? tool.inputSchema.required + : [] + } + }, + server: { + name: client.serverName, + icons: client.serverConfig.icons as string, + description: client.serverConfig.descriptions as string + } + }) + + // Populate the target map + nextToolNameToTargetMap.set(finalName, { + client, + originalName + }) } + } catch (error: unknown) { + if (options?.signal?.aborted) throw error + // Log error but continue building results from other clients + const errorMessage = error instanceof Error ? error.message : String(error) + const serverName = client.serverName || 'Unknown server' + console.error( + `Pass 2 Error: Error processing tools from server '${serverName}':`, + errorMessage + ) + this.serverManager.setServerLastError(serverName, errorMessage) + // Maybe skip adding tools from this client if listTools fails here again, + // though it succeeded in Pass 1. Or rely on the notification from Pass 1. + continue // Continue to next client } - } catch (error: unknown) { - // Log error but continue building results from other clients - const errorMessage = error instanceof Error ? error.message : String(error) - const serverName = client.serverName || 'Unknown server' - console.error( - `Pass 2 Error: Error processing tools from server '${serverName}':`, - errorMessage - ) - this.serverManager.setServerLastError(serverName, errorMessage) - // Maybe skip adding tools from this client if listTools fails here again, - // though it succeeded in Pass 1. Or rely on the notification from Pass 1. - continue // Continue to next client } - } - // Cache results and return - this.cachedToolDefinitions = results - console.info(`Cached ${results.length} final tool definitions and populated target map.`) + // Cache results and return + options?.signal?.throwIfAborted() + if (refreshGeneration !== this.toolDefinitionsCacheGeneration) { + if (this.activeToolDefinitionsRefresh === refresh) { + this.activeToolDefinitionsRefresh = null + } + return await this.getAllToolDefinitions(access, options) + } + this.cachedToolDefinitions = results + this.toolNameToTargetMap = nextToolNameToTargetMap + console.info(`Cached ${results.length} final tool definitions and populated target map.`) - return this.filterToolDefinitionsByContext(this.cachedToolDefinitions, context) + return this.filterToolDefinitionsByContext(this.cachedToolDefinitions, context) + } finally { + refresh.settle() + if (this.activeToolDefinitionsRefresh === refresh) { + this.activeToolDefinitionsRefresh = null + } + } } private filterToolDefinitionsByContext( @@ -460,7 +519,9 @@ export class ToolManager { */ async preCheckToolPermission( toolCall: MCPToolCall, - access?: Pick + access?: Pick & { + signal?: AbortSignal + } ): Promise<{ needsPermission: true toolName: string @@ -477,10 +538,15 @@ export class ToolManager { baseCommand?: string } } | null> { + access?.signal?.throwIfAborted() const finalName = toolCall.function.name // Ensure definitions and map are loaded/cached - await this.getAllToolDefinitions() + await awaitWithAbort( + this.getAllToolDefinitions(undefined, { signal: access?.signal }), + access?.signal + ) + access?.signal?.throwIfAborted() if (!this.toolNameToTargetMap) { console.error('[ToolManager] Tool target map is not available for permission check.') @@ -498,7 +564,8 @@ export class ToolManager { const toolServerName = targetInfo.client.serverName // Get server config to check auto-approve settings - const servers = await this.configPresenter.getMcpServers() + const servers = await awaitWithAbort(this.configPresenter.getMcpServers(), access?.signal) + access?.signal?.throwIfAborted() const serverConfig = servers[toolServerName] const accessContext = normalizeToolAccessContext({ agentId: access?.agentId, @@ -542,9 +609,12 @@ export class ToolManager { async callTool( toolCall: MCPToolCall, - access?: Pick + access?: Pick & { + signal?: AbortSignal + } ): Promise { try { + access?.signal?.throwIfAborted() const finalName = toolCall.function.name const argsString = toolCall.function.arguments @@ -556,7 +626,11 @@ export class ToolManager { }) // Ensure definitions and map are loaded/cached - await this.getAllToolDefinitions() + await awaitWithAbort( + this.getAllToolDefinitions(undefined, { signal: access?.signal }), + access?.signal + ) + access?.signal?.throwIfAborted() if (!this.toolNameToTargetMap) { console.error('Tool target map is not available.') @@ -592,12 +666,19 @@ export class ToolManager { // ACP agent-level MCP access control resolves from session context, not global chat mode. if (shouldResolveAcpContext && toolCall.conversationId) { try { - const acpContext = await this.resolveAcpSessionContext(toolCall.conversationId) + const acpContext = await awaitWithAbort( + this.resolveAcpSessionContext(toolCall.conversationId), + access?.signal + ) if (acpContext?.providerId === 'acp' && acpContext.agentId) { - const acpAgents = await this.configPresenter.getAcpAgents() + const acpAgents = await awaitWithAbort( + this.configPresenter.getAcpAgents(), + access?.signal + ) if (acpAgents.some((item) => item.id === acpContext.agentId)) { - const selections = await this.configPresenter.getAgentMcpSelections( - acpContext.agentId + const selections = await awaitWithAbort( + this.configPresenter.getAgentMcpSelections(acpContext.agentId), + access?.signal ) if (!selections?.length || !selections.includes(toolServerName)) { return { @@ -609,12 +690,14 @@ export class ToolManager { } } } catch (error) { + if (access?.signal?.aborted || isAbortError(error)) throw error console.warn( '[ToolManager] Failed to resolve ACP agent context for MCP access control:', error ) } } + access?.signal?.throwIfAborted() // Log the call details including original name console.info('[MCP] ToolManager calling tool', { @@ -644,7 +727,8 @@ export class ToolManager { } // Get server configuration - const servers = await this.configPresenter.getMcpServers() + const servers = await awaitWithAbort(this.configPresenter.getMcpServers(), access?.signal) + access?.signal?.throwIfAborted() const serverConfig = servers[toolServerName] if (!serverConfig) { console.error(`Configuration for server '${toolServerName}' not found.`) @@ -709,8 +793,10 @@ export class ToolManager { targetClient, serverConfig, originalName, - args || {} + args || {}, + access?.signal ) + access?.signal?.throwIfAborted() if (!preparedArgs.ok) { return { toolCallId: toolCall.id, @@ -720,7 +806,10 @@ export class ToolManager { } // Call the tool on the target client using the ORIGINAL name - const result = await targetClient.callTool(originalName, preparedArgs.args) + const result = access?.signal + ? await targetClient.callTool(originalName, preparedArgs.args, { signal: access.signal }) + : await targetClient.callTool(originalName, preparedArgs.args) + access?.signal?.throwIfAborted() // Format response let formattedContent: string | MCPContentItem[] = '' @@ -757,6 +846,10 @@ export class ToolManager { return response } catch (error: unknown) { + if (access?.signal?.aborted || isAbortError(error)) { + throw error + } + const errorMessage = error instanceof Error ? error.message : String(error) console.error('Unhandled error during tool call:', error) return { @@ -771,7 +864,8 @@ export class ToolManager { client: McpClient, serverConfig: MCPServerConfig, toolName: string, - args: Record + args: Record, + signal?: AbortSignal ): Promise<{ ok: true; args: Record } | { ok: false; error: string }> { if ( toolName !== 'launch_app' || @@ -781,12 +875,13 @@ export class ToolManager { return { ok: true, args } } - return await this.prepareCuaWindowsLaunchArgs(client, args) + return await this.prepareCuaWindowsLaunchArgs(client, args, signal) } private async prepareCuaWindowsLaunchArgs( client: McpClient, - args: Record + args: Record, + signal?: AbortSignal ): Promise<{ ok: true; args: Record } | { ok: false; error: string }> { const normalizedArgs = { ...args } const bundleId = this.readStringArg(normalizedArgs.bundle_id) @@ -818,7 +913,7 @@ export class ToolManager { return { ok: true, args: normalizedArgs } } - const apps = await this.listCuaWindowsApps(client) + const apps = await this.listCuaWindowsApps(client, signal) if (!apps) { return { ok: false, @@ -850,13 +945,19 @@ export class ToolManager { } private async listCuaWindowsApps( - client: McpClient + client: McpClient, + signal?: AbortSignal ): Promise> | null> { try { - const result = (await client.callTool('list_apps', {})) as { + const result = ( + signal + ? await client.callTool('list_apps', {}, { signal }) + : await client.callTool('list_apps', {}) + ) as { structuredContent?: unknown content?: unknown } + signal?.throwIfAborted() const structured = result.structuredContent if ( structured && @@ -871,6 +972,7 @@ export class ToolManager { return parsed.apps as Array> } } catch (error) { + if (signal?.aborted || isAbortError(error)) throw error console.warn('[MCP] Failed to preflight CUA Windows launch target:', error) } return null diff --git a/src/main/presenter/sqlitePresenter/tables/deepchatMessages.ts b/src/main/presenter/sqlitePresenter/tables/deepchatMessages.ts index fca92761a..e49057017 100644 --- a/src/main/presenter/sqlitePresenter/tables/deepchatMessages.ts +++ b/src/main/presenter/sqlitePresenter/tables/deepchatMessages.ts @@ -113,6 +113,12 @@ export class DeepChatMessagesTable extends BaseTable { .run(status, Date.now(), messageId) } + updateMetadata(messageId: string, metadata: string): void { + this.db + .prepare('UPDATE deepchat_messages SET metadata = ?, updated_at = ? WHERE id = ?') + .run(metadata, Date.now(), messageId) + } + incrementOrderSeqFrom(sessionId: string, fromOrderSeq: number): void { this.db .prepare( diff --git a/src/main/presenter/toolPresenter/agentTools/subagentOrchestratorTool.ts b/src/main/presenter/toolPresenter/agentTools/subagentOrchestratorTool.ts index db07f6b19..1dc9c0081 100644 --- a/src/main/presenter/toolPresenter/agentTools/subagentOrchestratorTool.ts +++ b/src/main/presenter/toolPresenter/agentTools/subagentOrchestratorTool.ts @@ -5,8 +5,13 @@ import type { DeepChatSubagentSlot } from '@shared/types/agent-interface' import type { AgentToolProgressUpdate } from '@shared/types/presenters/tool.presenter' import type { AgentToolCallResult } from './agentToolManager' import type { AgentToolRuntimePort, ConversationSessionInfo } from '../runtimePorts' +import { awaitWithAbort } from '@/lib/awaitWithAbort' export const SUBAGENT_ORCHESTRATOR_TOOL_NAME = 'subagent_orchestrator' +const DEFAULT_RUN_TIMEOUT_MS = 300000 +const MIN_RUN_TIMEOUT_MS = 1000 +const MAX_RUN_TIMEOUT_MS = 1800000 +const MAX_ACTIVE_RUNS_PER_PARENT = 3 const SUBAGENT_WORKDIR_RULE = 'Every child session inherits the same working directory as the parent session.' const SUBAGENT_PROMPT_DESCRIPTION = [ @@ -29,7 +34,8 @@ export const subagentOrchestratorSchema = z tasks: z.array(subagentOrchestratorTaskSchema).min(1).max(5).optional(), background: z.boolean().default(false).optional(), runId: z.string().trim().min(1).optional(), - timeoutMs: z.number().int().min(0).max(300000).optional() + timeoutMs: z.number().int().min(0).max(300000).optional(), + runTimeoutMs: z.number().int().min(MIN_RUN_TIMEOUT_MS).max(MAX_RUN_TIMEOUT_MS).optional() }) .superRefine((value, ctx) => { if (value.operation === 'run') { @@ -91,9 +97,13 @@ type MutableTaskState = { resultSummary?: string runtimeStatus?: 'idle' | 'generating' | 'error' started: boolean + handoffSettled: boolean cancelRequested: boolean tapeFinalized: boolean tapeFinalizeError?: string + tapeFinalizePromise?: Promise + cancellationPromise?: Promise + cancellationSettled: boolean completion: { promise: Promise resolve: () => void @@ -110,8 +120,13 @@ type MutableRunState = { status: SubagentTerminalStatus createdAt: number updatedAt: number + runTimeoutMs: number + deadlineAt: number completion: Promise abortController: AbortController + executionSettled: boolean + deadlineTimer?: ReturnType + cancellationReason?: string error?: string } @@ -147,6 +162,20 @@ const summarizeResult = (value: string): string | undefined => { const errorMessage = (error: unknown): string => error instanceof Error ? error.message : String(error) +const awaitWithSubagentCancellation = async ( + promise: Promise, + signal?: AbortSignal +): Promise => { + try { + return await awaitWithAbort(promise, signal) + } catch (error) { + if (signal?.aborted) { + throw new Error('subagent_orchestrator cancelled.') + } + throw error + } +} + const hasTapeFinalizeError = (tasks: MutableTaskState[]): boolean => tasks.some((task) => Boolean(task.tapeFinalizeError?.trim())) @@ -214,9 +243,18 @@ const buildHandoffMessage = (params: { task: MutableTaskState inheritedWorkspace: string | null }): string => { - const contract = - params.task.expectedOutput?.trim() || - 'Return a concise markdown result with your answer, key findings, and any important file paths or commands.' + const contract = [ + 'Return concise markdown with all of these sections:', + '## Result', + '## Evidence', + '## Changed Files', + '## Validation', + '## Unresolved', + 'Use `None` as the section content when a section has no entries.' + ] + const additionalRequirements = params.task.expectedOutput?.trim() + ? ['', 'Additional Requirements:', params.task.expectedOutput.trim()] + : [] return [ '# Structured Handoff', @@ -231,7 +269,8 @@ const buildHandoffMessage = (params: { params.task.prompt, '', 'Output Contract:', - contract, + ...contract, + ...additionalRequirements, '', 'Current Agent Working Directory:', params.inheritedWorkspace?.trim() || '(none)', @@ -275,7 +314,7 @@ export class SubagentOrchestratorTool { } private updateRunStatus(run: MutableRunState): void { - run.status = this.resolveRunStatus(run.tasks) + run.status = run.cancellationReason ? 'cancelled' : this.resolveRunStatus(run.tasks) run.updatedAt = Date.now() } @@ -288,6 +327,9 @@ export class SubagentOrchestratorTool { status: run.status, createdAt: run.createdAt, updatedAt: run.updatedAt, + runTimeoutMs: run.runTimeoutMs, + deadlineAt: run.deadlineAt, + cancellationReason: run.cancellationReason, error: run.error, tasks: run.tasks.map((task) => ({ taskId: task.taskId, @@ -390,16 +432,127 @@ export class SubagentOrchestratorTool { } } + private markTaskCancelled(task: MutableTaskState, reason: string): void { + task.cancelRequested = true + task.status = 'cancelled' + task.resultSummary = task.resultSummary || reason + task.updatedAt = Date.now() + task.completion.resolve() + } + + private isTaskCancellationRequested( + run: MutableRunState, + task: MutableTaskState, + parentSignal?: AbortSignal + ): boolean { + return ( + parentSignal?.aborted === true || run.abortController.signal.aborted || task.cancelRequested + ) + } + + private requestTaskCancellation( + task: MutableTaskState, + forceNewRequest = false + ): Promise | undefined { + const childSessionId = task.sessionId + if (!childSessionId) { + return undefined + } + + if (task.cancellationPromise && !forceNewRequest) { + return task.cancellationPromise + } + + const previousCancellation = task.cancellationPromise + const currentCancellation = (async () => { + try { + await this.runtimePort.cancelConversation(childSessionId) + } catch { + // Cancellation is best effort, but tape finalization must still observe its settlement. + } + })() + + task.cancellationSettled = false + let combinedCancellation: Promise + combinedCancellation = Promise.all( + previousCancellation ? [previousCancellation, currentCancellation] : [currentCancellation] + ) + .then(() => undefined) + .finally(() => { + if (task.cancellationPromise === combinedCancellation) { + task.cancellationSettled = true + } + }) + task.cancellationPromise = combinedCancellation + return combinedCancellation + } + + private async cancelRun(run: MutableRunState, reason: string): Promise { + run.abortController.abort() + if (!run.executionSettled) { + run.cancellationReason = run.cancellationReason || reason + } + + const cancellationRequests: Promise[] = [] + for (const task of run.tasks) { + if (isTerminalStatus(task.status)) { + continue + } + + this.markTaskCancelled(task, reason) + + const cancellationRequest = this.requestTaskCancellation(task) + if (cancellationRequest) { + cancellationRequests.push(cancellationRequest) + } + } + + this.updateRunStatus(run) + await Promise.all(cancellationRequests) + } + + private async cancelAndFinalizeTask(params: { + run: MutableRunState + task: MutableTaskState + reason: string + forceNewCancellation?: boolean + }): Promise { + const { run, task, reason, forceNewCancellation = false } = params + this.markTaskCancelled(task, reason) + + const cancellationRequest = this.requestTaskCancellation(task, forceNewCancellation) + if (cancellationRequest) { + await cancellationRequest + } + + await this.finalizeTaskTape({ + parentSessionId: run.parentSessionId, + runId: run.runId, + task + }) + } + private async finalizeTaskTape(params: { parentSessionId: string runId: string task: MutableTaskState }): Promise { const { parentSessionId, runId, task } = params - if (!task.sessionId || task.tapeFinalized) { + if ( + !task.sessionId || + task.tapeFinalized || + (task.status === 'cancelled' && + (!task.handoffSettled || (task.cancellationPromise && !task.cancellationSettled))) + ) { return } + if (task.tapeFinalizePromise) { + await task.tapeFinalizePromise + return + } + + const childSessionId = task.sessionId const meta = { runId, taskId: task.taskId, @@ -409,23 +562,29 @@ export class SubagentOrchestratorTool { resultSummary: task.resultSummary ?? null } - try { - if (task.status === 'completed') { - await this.runtimePort.mergeSubagentTape?.(parentSessionId, task.sessionId, meta) - } else { - await this.runtimePort.discardSubagentTape?.(parentSessionId, task.sessionId, meta) + task.tapeFinalizePromise = (async () => { + try { + if (task.status === 'completed') { + await this.runtimePort.mergeSubagentTape?.(parentSessionId, childSessionId, meta) + } else { + await this.runtimePort.discardSubagentTape?.(parentSessionId, childSessionId, meta) + } + task.tapeFinalized = true + task.tapeFinalizeError = undefined + } catch (error) { + task.tapeFinalizeError = errorMessage(error) + console.warn('[SubagentOrchestratorTool] Failed to finalize subagent tape fork:', { + parentSessionId, + childSessionId: task.sessionId, + status: task.status, + error + }) + } finally { + task.tapeFinalizePromise = undefined } - task.tapeFinalized = true - task.tapeFinalizeError = undefined - } catch (error) { - task.tapeFinalizeError = errorMessage(error) - console.warn('[SubagentOrchestratorTool] Failed to finalize subagent tape fork:', { - parentSessionId, - childSessionId: task.sessionId, - status: task.status, - error - }) - } + })() + + await task.tapeFinalizePromise } private async retryPendingTapeFinalization(run: MutableRunState): Promise { @@ -438,11 +597,26 @@ export class SubagentOrchestratorTool { continue } - await this.finalizeTaskTape({ + const finalization = { parentSessionId: run.parentSessionId, runId: run.runId, task - }) + } + + if (!run.executionSettled) { + if (task.status !== 'cancelled' || !task.cancellationSettled) { + continue + } + + // Cancellation settlement makes discard safe, but a slow tape backend must not turn the + // run deadline into another unbounded wait for foreground or polling callers. + void this.finalizeTaskTape(finalization) + .then(() => this.updateRunStatus(run)) + .catch(() => undefined) + continue + } + + await this.finalizeTaskTape(finalization) } this.updateRunStatus(run) @@ -479,23 +653,7 @@ export class SubagentOrchestratorTool { const run = this.getRunForSession(conversationId, args.runId) if (args.operation === 'kill') { - run.abortController.abort() - for (const task of run.tasks) { - if (isTerminalStatus(task.status)) { - continue - } - - task.cancelRequested = true - task.status = 'cancelled' - task.resultSummary = task.resultSummary || 'Cancelled by parent session.' - task.updatedAt = Date.now() - task.completion.resolve() - - if (task.sessionId) { - await this.runtimePort.cancelConversation(task.sessionId).catch(() => undefined) - } - } - this.updateRunStatus(run) + await this.cancelRun(run, 'Cancelled by parent session.') return this.buildRunProgressResult(run, 'Subagent run cancelled') } @@ -531,19 +689,20 @@ export class SubagentOrchestratorTool { timeoutMs: number, signal?: AbortSignal ): Promise { + if (signal?.aborted) { + throw new Error('subagent_orchestrator cancelled.') + } + let abortListener: (() => void) | undefined + let timeout: ReturnType | undefined const pending = [ run.completion, new Promise((resolve) => { - setTimeout(resolve, timeoutMs) + timeout = setTimeout(resolve, timeoutMs) }) ] if (signal) { - if (signal.aborted) { - throw new Error('subagent_orchestrator cancelled.') - } - pending.push( new Promise((_, reject) => { abortListener = () => { @@ -557,6 +716,7 @@ export class SubagentOrchestratorTool { try { await Promise.race(pending) } finally { + if (timeout !== undefined) clearTimeout(timeout) if (signal && abortListener) { signal.removeEventListener('abort', abortListener) } @@ -690,7 +850,7 @@ export class SubagentOrchestratorTool { expectedOutput: { type: 'string', description: - 'Optional output contract for the child session, such as structure, scope, or formatting requirements.' + 'Optional requirements appended to the standard child output contract.' } }, required: ['slotId', 'title', 'prompt'] @@ -708,6 +868,13 @@ export class SubagentOrchestratorTool { timeoutMs: { type: 'number', description: 'Maximum wait time for operation=wait. Defaults to 60000.' + }, + runTimeoutMs: { + type: 'number', + minimum: MIN_RUN_TIMEOUT_MS, + maximum: MAX_RUN_TIMEOUT_MS, + description: + 'Maximum lifetime for operation=run, independent of wait timeout. Defaults to 300000.' } } } @@ -734,7 +901,14 @@ export class SubagentOrchestratorTool { throw new Error('subagent_orchestrator requires a conversationId.') } - const parent = await this.runtimePort.resolveConversationSessionInfo(conversationId) + if (options?.signal?.aborted) { + throw new Error('subagent_orchestrator cancelled.') + } + + const parent = await awaitWithSubagentCancellation( + this.runtimePort.resolveConversationSessionInfo(conversationId), + options?.signal + ) if (!parent) { throw new Error(`Conversation not found: ${conversationId}`) } @@ -756,12 +930,27 @@ export class SubagentOrchestratorTool { const mode = args.mode ?? 'parallel' const taskSpecs = args.tasks ?? [] const inheritedWorkspace = - (await this.runtimePort.resolveConversationWorkdir(parent.sessionId))?.trim() || + ( + await awaitWithSubagentCancellation( + this.runtimePort.resolveConversationWorkdir(parent.sessionId), + options?.signal + ) + )?.trim() || parent.projectDir?.trim() || null + const activeRunCount = [...this.runs.values()].filter( + (run) => run.parentSessionId === conversationId && !isTerminalStatus(run.status) + ).length + if (activeRunCount >= MAX_ACTIVE_RUNS_PER_PARENT) { + throw new Error( + `A parent session can have at most ${MAX_ACTIVE_RUNS_PER_PARENT} active subagent runs.` + ) + } + const slotMap = new Map(parent.availableSubagentSlots.map((slot) => [slot.id, slot])) const now = Date.now() + const runTimeoutMs = args.runTimeoutMs ?? DEFAULT_RUN_TIMEOUT_MS const tasks = taskSpecs.map((task, index): MutableTaskState => { const slot = slotMap.get(task.slotId) if (!slot) { @@ -790,8 +979,10 @@ export class SubagentOrchestratorTool { updatedAt: now, waitingInteraction: null, started: false, + handoffSettled: false, cancelRequested: false, tapeFinalized: false, + cancellationSettled: false, completion: createDeferred() } }) @@ -810,14 +1001,18 @@ export class SubagentOrchestratorTool { status: 'queued', createdAt: now, updatedAt: now, + runTimeoutMs, + deadlineAt: now + runTimeoutMs, completion: Promise.resolve(), - abortController + abortController, + executionSettled: false } this.runs.set(runId, run) + let lifecycleSettled = false const emitProgress = () => { this.updateRunStatus(run) - if (!options?.onProgress || run.background) { + if (lifecycleSettled || !options?.onProgress || run.background) { return } @@ -894,33 +1089,38 @@ export class SubagentOrchestratorTool { emitProgress() }) + let resolveParentCancellation: (() => void) | undefined + const parentCancellation = options?.signal + ? new Promise((resolve) => { + resolveParentCancellation = resolve + }) + : undefined + let parentCancellationObserved = false const abortListener = () => { - abortController.abort() - for (const task of tasks) { - if (isTerminalStatus(task.status)) { - continue - } - - task.cancelRequested = true - task.updatedAt = Date.now() - updateTaskStatusFromRuntime(task) - - if (task.sessionId) { - void this.runtimePort.cancelConversation(task.sessionId).catch(() => undefined) - } + if (parentCancellationObserved) { + return } + parentCancellationObserved = true + void this.cancelRun(run, 'Cancelled by parent session.') emitProgress() + resolveParentCancellation?.() } - options?.signal?.addEventListener('abort', abortListener) + options?.signal?.addEventListener('abort', abortListener, { once: true }) + if (options?.signal?.aborted) { + abortListener() + } const runTask = async (task: MutableTaskState): Promise => { - if (options?.signal?.aborted || abortController.signal.aborted) { + if (options?.signal?.aborted) { abortListener() + } + if (this.isTaskCancellationRequested(run, task, options?.signal)) { return } + let handoffAttempted = false try { const child = await this.runtimePort.createSubagentSession({ parentSessionId: parent.sessionId, @@ -946,17 +1146,12 @@ export class SubagentOrchestratorTool { task.updatedAt = Date.now() sessionTaskMap.set(child.sessionId, task) - if (options?.signal?.aborted || abortController.signal.aborted || task.cancelRequested) { - task.cancelRequested = true - task.updatedAt = Date.now() - task.status = 'cancelled' - task.resultSummary = task.resultSummary || 'Cancelled by parent session.' - maybeResolveTask(task) - await this.runtimePort.cancelConversation(child.sessionId).catch(() => undefined) - await this.finalizeTaskTape({ - parentSessionId: parent.sessionId, - runId, - task + if (this.isTaskCancellationRequested(run, task, options?.signal)) { + task.handoffSettled = true + await this.cancelAndFinalizeTask({ + run, + task, + reason: run.cancellationReason || 'Cancelled by parent session.' }) emitProgress() return @@ -971,7 +1166,24 @@ export class SubagentOrchestratorTool { task, inheritedWorkspace }) + handoffAttempted = true await this.runtimePort.sendConversationMessage(child.sessionId, handoff) + task.handoffSettled = true + + if (options?.signal?.aborted) { + abortListener() + } + if (this.isTaskCancellationRequested(run, task, options?.signal)) { + await this.cancelAndFinalizeTask({ + run, + task, + reason: run.cancellationReason || 'Cancelled by parent session.', + forceNewCancellation: true + }) + emitProgress() + return + } + task.started = true task.updatedAt = Date.now() if (task.status === 'queued') { @@ -980,6 +1192,7 @@ export class SubagentOrchestratorTool { emitProgress() await task.completion.promise + await task.cancellationPromise await this.finalizeTaskTape({ parentSessionId: parent.sessionId, runId, @@ -987,20 +1200,34 @@ export class SubagentOrchestratorTool { }) } catch (error) { task.updatedAt = Date.now() - task.status = task.cancelRequested ? 'cancelled' : 'error' - task.resultSummary = - error instanceof Error ? error.message : 'Subagent session failed unexpectedly.' - maybeResolveTask(task) - await this.finalizeTaskTape({ - parentSessionId: parent.sessionId, - runId, - task - }) + task.handoffSettled = true + if (options?.signal?.aborted) { + abortListener() + } + + if (this.isTaskCancellationRequested(run, task, options?.signal)) { + await this.cancelAndFinalizeTask({ + run, + task, + reason: run.cancellationReason || 'Cancelled by parent session.', + forceNewCancellation: handoffAttempted + }) + } else { + task.status = 'error' + task.resultSummary = + error instanceof Error ? error.message : 'Subagent session failed unexpectedly.' + maybeResolveTask(task) + await this.finalizeTaskTape({ + parentSessionId: parent.sessionId, + runId, + task + }) + } emitProgress() } } - const runCompletion = (async () => { + const execution = (async () => { emitProgress() try { @@ -1009,7 +1236,6 @@ export class SubagentOrchestratorTool { } else { for (const task of tasks) { if (abortController.signal.aborted) { - abortListener() break } await runTask(task) @@ -1026,14 +1252,35 @@ export class SubagentOrchestratorTool { task.updatedAt = Date.now() task.completion.resolve() } - } finally { - this.updateRunStatus(run) - emitProgress() - unsubscribe() - options?.signal?.removeEventListener('abort', abortListener) - this.pruneRuns() } - })() + })().finally(() => { + run.executionSettled = true + }) + + const deadline = new Promise((resolve) => { + run.deadlineTimer = setTimeout(() => { + const reason = `Run deadline exceeded after ${run.runTimeoutMs}ms.` + void this.cancelRun(run, reason) + resolve() + }, run.runTimeoutMs) + }) + const lifecycleEvents = [execution, deadline] + if (parentCancellation) { + lifecycleEvents.push(parentCancellation) + } + const runCompletion = Promise.race(lifecycleEvents).finally(() => { + if (run.deadlineTimer !== undefined) { + clearTimeout(run.deadlineTimer) + run.deadlineTimer = undefined + } + this.updateRunStatus(run) + emitProgress() + lifecycleSettled = true + unsubscribe() + options?.signal?.removeEventListener('abort', abortListener) + this.pruneRuns() + }) + run.completion = runCompletion void runCompletion.catch(() => undefined) diff --git a/src/main/presenter/toolPresenter/index.ts b/src/main/presenter/toolPresenter/index.ts index d2333aa28..524df45eb 100644 --- a/src/main/presenter/toolPresenter/index.ts +++ b/src/main/presenter/toolPresenter/index.ts @@ -1,3 +1,4 @@ +import { awaitWithAbort } from '@/lib/awaitWithAbort' import type { IConfigPresenter, IMCPPresenter, @@ -45,7 +46,7 @@ export interface IToolPresenter { ): Promise<{ content: unknown; rawData: MCPToolResponse }> preCheckToolPermission?( request: MCPToolCall, - options?: { permissionMode?: PermissionMode } + options?: { permissionMode?: PermissionMode; signal?: AbortSignal } ): Promise clearConversationToolMapping?(conversationId: string): void clearAgentPlanState?(conversationId: string): void @@ -317,7 +318,8 @@ export class ToolPresenter implements IToolPresenter { const storedAccess = this.getConversationMcpAccessContext(request.conversationId) return await this.options.mcpPresenter.callTool(request, { agentId: options?.agentId ?? storedAccess?.agentId, - enabledServerIds: options?.enabledMcpServerIds ?? storedAccess?.enabledMcpServerIds + enabledServerIds: options?.enabledMcpServerIds ?? storedAccess?.enabledMcpServerIds, + signal: options?.signal }) } @@ -327,8 +329,9 @@ export class ToolPresenter implements IToolPresenter { */ async preCheckToolPermission( request: MCPToolCall, - options?: { permissionMode?: PermissionMode } + options?: { permissionMode?: PermissionMode; signal?: AbortSignal } ): Promise { + options?.signal?.throwIfAborted() const toolName = request.function.name const source = this.getToolSource(toolName, request.conversationId) @@ -365,13 +368,11 @@ export class ToolPresenter implements IToolPresenter { } } - const result = await this.agentToolManager.preCheckToolPermission( - toolName, - args, - request.conversationId, - { + const result = await awaitWithAbort( + this.agentToolManager.preCheckToolPermission(toolName, args, request.conversationId, { allowExternalFileAccess: allowsExternalFileAccess(options?.permissionMode) - } + }), + options?.signal ) if (!result) { return null @@ -384,7 +385,8 @@ export class ToolPresenter implements IToolPresenter { const storedAccess = this.getConversationMcpAccessContext(request.conversationId) return await this.options.mcpPresenter.preCheckToolPermission(request, { agentId: storedAccess?.agentId, - enabledServerIds: storedAccess?.enabledMcpServerIds + enabledServerIds: storedAccess?.enabledMcpServerIds, + signal: options?.signal }) } diff --git a/src/renderer/src/components/chat/messageListItems.ts b/src/renderer/src/components/chat/messageListItems.ts index c5e2aa62f..e8fc1a834 100644 --- a/src/renderer/src/components/chat/messageListItems.ts +++ b/src/renderer/src/components/chat/messageListItems.ts @@ -95,6 +95,7 @@ export type DisplayAssistantMessageExtra = Record diff --git a/test/main/agent/deepchat/instance/deepChatAgentRuntime.test.ts b/test/main/agent/deepchat/instance/deepChatAgentRuntime.test.ts index e1a6c645c..3f4b554f6 100644 --- a/test/main/agent/deepchat/instance/deepChatAgentRuntime.test.ts +++ b/test/main/agent/deepchat/instance/deepChatAgentRuntime.test.ts @@ -167,7 +167,7 @@ describe('DeepChatAgentRuntime', () => { await expect(first.waitForFirstTurnReady()).resolves.toBe(true) }) - it('owns pre-stream and active generation state without stale-run clears', () => { + it('reuses an owned preparation controller for the active generation', () => { const runtime = new DeepChatAgentRuntime(() => createDelegate()) const instance = runtime.getOrHydrate(toAppSessionId('session')) const controller = new AbortController() @@ -178,6 +178,7 @@ describe('DeepChatAgentRuntime', () => { const run = createRun('session', 'run-1', 'message-1', controller) const generation = instance.registerActiveGeneration(run) expect(generation).toBe(run) + expect(controller.signal.aborted).toBe(false) expect(instance.getActiveGeneration()).toBe(generation) expect(instance.clearActiveGeneration('stale-run')).toBe(false) expect(instance.isActiveRun('run-1')).toBe(true) @@ -186,6 +187,38 @@ describe('DeepChatAgentRuntime', () => { expect(instance.getAbortController()).toBeUndefined() }) + it('aborts the previous active generation before replacing it', () => { + const runtime = new DeepChatAgentRuntime(() => createDelegate()) + const instance = runtime.getOrHydrate(toAppSessionId('session')) + const firstController = new AbortController() + const replacementController = new AbortController() + const replacement = createRun('session', 'run-2', 'message-2', replacementController) + + instance.registerActiveGeneration(createRun('session', 'run-1', 'message-1', firstController)) + instance.registerActiveGeneration(replacement) + + expect(firstController.signal.aborted).toBe(true) + expect(replacementController.signal.aborted).toBe(false) + expect(instance.getActiveGeneration()).toBe(replacement) + expect(instance.getAbortController()).toBe(replacementController) + }) + + it('aborts an owned preparation controller when a different active run replaces it', () => { + const runtime = new DeepChatAgentRuntime(() => createDelegate()) + const instance = runtime.getOrHydrate(toAppSessionId('session')) + const preparationController = new AbortController() + const activeController = new AbortController() + const run = createRun('session', 'run-1', 'message-1', activeController) + + instance.setAbortController(preparationController) + instance.registerActiveGeneration(run) + + expect(preparationController.signal.aborted).toBe(true) + expect(activeController.signal.aborted).toBe(false) + expect(instance.getActiveGeneration()).toBe(run) + expect(instance.getAbortController()).toBe(activeController) + }) + it('aborts pre-stream work immediately but retains an active run until settlement', () => { const runtime = new DeepChatAgentRuntime(() => createDelegate()) const first = runtime.getOrHydrate(toAppSessionId('first')) diff --git a/test/main/agent/deepchat/loop/contextCoordinator.test.ts b/test/main/agent/deepchat/loop/contextCoordinator.test.ts index a86474b3a..e476a0f77 100644 --- a/test/main/agent/deepchat/loop/contextCoordinator.test.ts +++ b/test/main/agent/deepchat/loop/contextCoordinator.test.ts @@ -341,6 +341,30 @@ describe('DeepChatContextCoordinator', () => { }) }) + it('checks retry availability before context recovery or another manifest', async () => { + const fixture = createAttemptInput({ + providerEvents: [[{ type: 'error', error_message: 'context overflow' }]] + }) + const limitError = new Error('provider attempt limit reached') + fixture.input.provider.assertAvailable = vi + .fn() + .mockImplementationOnce(() => undefined) + .mockImplementationOnce(() => { + throw limitError + }) + + await expect( + collect(new DeepChatContextCoordinator().streamProviderAttempts(fixture.input)) + ).rejects.toBe(limitError) + + expect(fixture.input.provider.assertAvailable).toHaveBeenCalledTimes(2) + expect(fixture.input.recovery.recover).not.toHaveBeenCalled() + expect(fixture.providerRequests).toHaveLength(1) + expect(fixture.manifests).toHaveLength(1) + expect(fixture.order.filter((entry) => entry === 'rate')).toHaveLength(1) + expect(fixture.run.requestSeq).toBe(1) + }) + it('runs pressure recovery before manifesting the provider request', async () => { const fixture = createAttemptInput() const preflight = vi diff --git a/test/main/agent/deepchat/loop/deepChatLoopEngine.test.ts b/test/main/agent/deepchat/loop/deepChatLoopEngine.test.ts index ed69acd5a..ab637a138 100644 --- a/test/main/agent/deepchat/loop/deepChatLoopEngine.test.ts +++ b/test/main/agent/deepchat/loop/deepChatLoopEngine.test.ts @@ -158,6 +158,35 @@ describe('DeepChatLoopEngine', () => { expect(executeToolBatch).not.toHaveBeenCalled() }) + it('includes previously executed tools when enforcing the global tool-call limit', async () => { + const run = createRun() + const executeToolBatch = vi.fn(async () => ({ + type: 'continue' as const, + executedToolCount: 1 + })) + + const outcome = await new DeepChatLoopEngine().run( + run, + { + initialExecutedToolCount: 128, + consumeProviderRound: async () => ({ + type: 'tool_batch' as const, + batch: 'resumed-overflow', + toolCallCount: 1 + }), + executeToolBatch + }, + createCommitCallbacks() + ) + + expect(outcome).toEqual({ + type: 'max_tool_calls', + attemptedToolCount: 129, + limit: 128 + }) + expect(executeToolBatch).not.toHaveBeenCalled() + }) + it('propagates a paused tool batch without entering another provider round', async () => { const run = createRun() const order: string[] = [] diff --git a/test/main/agent/deepchat/memory/memoryRuntimeCoordinator.test.ts b/test/main/agent/deepchat/memory/memoryRuntimeCoordinator.test.ts index d2d8a7674..057158ecf 100644 --- a/test/main/agent/deepchat/memory/memoryRuntimeCoordinator.test.ts +++ b/test/main/agent/deepchat/memory/memoryRuntimeCoordinator.test.ts @@ -396,6 +396,16 @@ describe('MemoryRuntimeCoordinator', () => { expect(deps.getTapeRows).toHaveBeenCalledTimes(2) }) + it.each([ + ['a JSON string', JSON.stringify(' Remember Redis. ')], + ['non-JSON plain text', ' Remember Redis. '] + ])('reads the latest user query from %s content', (_format, content) => { + const { coordinator, setRows } = createHarness() + setRows([{ ...createRecord('u1', 1, ''), content }]) + + expect(coordinator.getLatestUserQuery('s1')).toBe('Remember Redis.') + }) + it('dedupes non-null injection access, keeps null access, expires turns and clears on destroy', () => { const { coordinator, port } = createHarness() const now = vi.spyOn(Date, 'now').mockReturnValue(1_000) diff --git a/test/main/evals/nativeAgent/harness.ts b/test/main/evals/nativeAgent/harness.ts new file mode 100644 index 000000000..d2f55b39b --- /dev/null +++ b/test/main/evals/nativeAgent/harness.ts @@ -0,0 +1,678 @@ +import { vi } from 'vitest' +import { ModelType } from '@shared/model' +import type { AssistantMessageBlock } from '@shared/types/agent-interface' +import type { ChatMessage } from '@shared/types/core/chat-message' +import type { LLMCoreStreamEvent } from '@shared/types/core/llm-events' +import type { MCPToolCall, MCPToolDefinition } from '@shared/types/core/mcp' +import type { IToolPresenter } from '@shared/types/presenters/tool.presenter' +import type { DeepChatMessageStore } from '@/presenter/agentRuntimePresenter/messageStore' +import { processStream } from '@/presenter/agentRuntimePresenter/process' +import { ToolOutputGuard } from '@/presenter/agentRuntimePresenter/toolOutputGuard' +import { + createToolExecutionPort, + createToolResultPort +} from '@/presenter/agentRuntimePresenter/toolAdapters' +import { createState } from '@/presenter/agentRuntimePresenter/types' +import type { ProcessParams, ProcessResult } from '@/presenter/agentRuntimePresenter/types' +import { createLoopRun } from '@/agent/deepchat/loop/loopRun' +import { toAppSessionId } from '@/agent/shared/agentSessionIds' + +vi.mock('@/routes/publishDeepchatEvent', () => ({ + publishDeepchatEvent: vi.fn() +})) + +vi.mock('@/eventbus', () => ({ + eventBus: {} +})) + +vi.mock('@/events', () => ({ + STREAM_EVENTS: { + RESPONSE: 'stream:response', + END: 'stream:end', + ERROR: 'stream:error' + } +})) + +vi.mock('@/presenter', () => ({ + presenter: { + commandPermissionService: { + extractCommandSignature: vi.fn(() => 'eval-signature'), + approve: vi.fn() + }, + filePermissionService: { approve: vi.fn() }, + settingsPermissionService: { approve: vi.fn() }, + mcpPresenter: { grantPermission: vi.fn(async () => undefined) } + } +})) + +export interface ScriptedProviderRound { + events: LLMCoreStreamEvent[] + abortBeforeEventIndex?: number +} + +export interface ScriptedToolBehavior { + response?: string + error?: string + permission?: { + permissionType: 'read' | 'write' | 'all' | 'command' + description: string + } +} + +export type EvalPersistedStatus = 'none' | 'pending' | 'sent' | 'error' + +export interface ScriptedProviderUsage { + inputTokens: number + outputTokens: number + totalTokens: number + cachedInputTokens?: number + cacheWriteInputTokens?: number +} + +export interface NativeAgentEvalUsage { + inputTokens: number | null + outputTokens: number | null + totalTokens: number | null + cachedInputTokens: number | null + cacheWriteInputTokens: number | null +} + +export interface NativeAgentEvalExpected { + status: ProcessResult['status'] + stopReason: string | null + persistedStatus: EvalPersistedStatus + persistedRunOutcome: ProcessResult['status'] + persistedRunStopReason: string + providerRounds: number + toolCalls: number + finalTextIncludes?: string + terminalErrorIncludes?: string + errorMessageIncludes?: string + toolMessageIncludes?: string[] + failedToolCalls?: number + permissionRequests?: number + usage: NativeAgentEvalUsage +} + +export interface NativeAgentEvalScenario { + id: string + rounds: ScriptedProviderRound[] + tools?: Record + permissionMode?: ProcessParams['permissionMode'] + maxProviderRounds?: number + yieldForPendingInput?: boolean + budget: { + maxProviderRounds: number + maxToolCalls: number + } + expected: NativeAgentEvalExpected +} + +export interface NativeAgentEvalToolCall { + id: string + name: string + arguments: string + status: 'success' | 'error' +} + +export interface NativeAgentEvalReport { + schemaVersion: 1 + scenarioId: string + passed: boolean + outcome: { + status: ProcessResult['status'] + stopReason: string | null + terminalError: string | null + errorMessage: string | null + } + persistedStatus: EvalPersistedStatus + persistedRunId: string | null + persistedRunOutcome: string | null + persistedRunStopReason: string | null + persistedProviderRounds: number | null + persistedToolCalls: number | null + persistedUsage: NativeAgentEvalUsage + providerRounds: number + toolCalls: { + total: number + succeeded: number + failed: number + } + permissionRequests: number + elapsedMs: number + usage: NativeAgentEvalUsage + finalText: string + expectationFailures: string[] +} + +export interface NativeAgentEvalAggregate { + schemaVersion: 1 + scenarios: number + passed: number + passRate: number + totalProviderRounds: number + providerRoundBudget: number + totalToolCalls: number + toolCallBudget: number + totalTokens: number + withinCallBudgets: boolean +} + +interface EvalMessageStoreState { + persistedStatus: EvalPersistedStatus + blocks: AssistantMessageBlock[] + metadata: Record +} + +interface ToolPresenterHarness { + presenter: IToolPresenter + calls: NativeAgentEvalToolCall[] +} + +const DEFAULT_INTERLEAVED_REASONING = { + preserveReasoningContent: false, + forcedBySessionSetting: false, + portraitInterleaved: false, + reasoningSupported: false, + providerDbSourceUrl: 'https://example.com/provider-db.json' +} as const + +function clone(value: T): T { + return JSON.parse(JSON.stringify(value)) as T +} + +function createMessageStore(): { + messageStore: DeepChatMessageStore + state: EvalMessageStoreState +} { + const state: EvalMessageStoreState = { + persistedStatus: 'none', + blocks: [], + metadata: {} + } + + const captureMetadata = (rawMetadata: string | undefined): void => { + if (!rawMetadata) return + try { + const metadata = JSON.parse(rawMetadata) as unknown + if (metadata && typeof metadata === 'object' && !Array.isArray(metadata)) { + state.metadata = metadata as Record + } + } catch { + state.metadata = {} + } + } + + const messageStore = { + addSearchResult: vi.fn(), + getMessage: vi.fn(() => null), + updateAssistantContent: vi.fn( + (_messageId: string, blocks: AssistantMessageBlock[], metadata?: string) => { + if (state.persistedStatus === 'none') { + state.persistedStatus = 'pending' + } + state.blocks = clone(blocks) + captureMetadata(metadata) + } + ), + finalizeAssistantMessage: vi.fn( + (_messageId: string, blocks: AssistantMessageBlock[], metadata: string) => { + state.persistedStatus = 'sent' + state.blocks = clone(blocks) + captureMetadata(metadata) + } + ), + setMessageError: vi.fn( + (_messageId: string, blocks: AssistantMessageBlock[], metadata?: string) => { + state.persistedStatus = 'error' + state.blocks = clone(blocks) + captureMetadata(metadata) + } + ) + } + + return { + messageStore: messageStore as unknown as DeepChatMessageStore, + state + } +} + +function makeToolDefinition(name: string): MCPToolDefinition { + return { + type: 'function', + source: 'agent', + function: { + name, + description: `Eval tool ${name}`, + parameters: { type: 'object', properties: {} } + }, + server: { + name: 'native-agent-eval', + icons: '', + description: 'Native Agent deterministic evaluation tools' + } + } +} + +function createToolPresenter( + behaviors: Record +): ToolPresenterHarness { + const calls: NativeAgentEvalToolCall[] = [] + + const presenter = { + getAllToolDefinitions: vi.fn(async () => Object.keys(behaviors).map(makeToolDefinition)), + preCheckToolPermission: vi.fn(async (request: MCPToolCall) => { + const permission = behaviors[request.function.name]?.permission + if (!permission) return null + + return { + needsPermission: true as const, + toolName: request.function.name, + serverName: 'native-agent-eval', + permissionType: permission.permissionType, + description: permission.description + } + }), + callTool: vi.fn(async (request: MCPToolCall) => { + const behavior = behaviors[request.function.name] ?? {} + const call: NativeAgentEvalToolCall = { + id: request.id, + name: request.function.name, + arguments: request.function.arguments, + status: 'success' + } + calls.push(call) + + if (behavior.error) { + call.status = 'error' + throw new Error(behavior.error) + } + + const content = behavior.response ?? `result for ${request.function.name}` + return { + content, + rawData: { + toolCallId: request.id, + content, + isError: false + } + } + }), + buildToolSystemPrompt: vi.fn(() => '') + } + + return { + presenter: presenter as unknown as IToolPresenter, + calls + } +} + +function stringifyMessageContent(message: ChatMessage): string { + return typeof message.content === 'string' ? message.content : JSON.stringify(message.content) +} + +function collectFinalText(blocks: AssistantMessageBlock[]): string { + return blocks + .map((block) => (typeof block.content === 'string' ? block.content : '')) + .filter(Boolean) + .join('\n') +} + +function normalizeUsage(result: ProcessResult): NativeAgentEvalUsage { + return { + inputTokens: typeof result.usage?.inputTokens === 'number' ? result.usage.inputTokens : null, + outputTokens: typeof result.usage?.outputTokens === 'number' ? result.usage.outputTokens : null, + totalTokens: typeof result.usage?.totalTokens === 'number' ? result.usage.totalTokens : null, + cachedInputTokens: + typeof result.usage?.cachedInputTokens === 'number' ? result.usage.cachedInputTokens : null, + cacheWriteInputTokens: + typeof result.usage?.cacheWriteInputTokens === 'number' + ? result.usage.cacheWriteInputTokens + : null + } +} + +function normalizePersistedNumber(metadata: Record, key: string): number | null { + const value = metadata[key] + return typeof value === 'number' && Number.isFinite(value) ? value : null +} + +function normalizePersistedUsage(metadata: Record): NativeAgentEvalUsage { + return { + inputTokens: normalizePersistedNumber(metadata, 'inputTokens'), + outputTokens: normalizePersistedNumber(metadata, 'outputTokens'), + totalTokens: normalizePersistedNumber(metadata, 'totalTokens'), + cachedInputTokens: normalizePersistedNumber(metadata, 'cachedInputTokens'), + cacheWriteInputTokens: normalizePersistedNumber(metadata, 'cacheWriteInputTokens') + } +} + +function addMismatch(failures: string[], label: string, actual: unknown, expected: unknown): void { + if (actual !== expected) { + failures.push(`${label}: expected ${String(expected)}, received ${String(actual)}`) + } +} + +function evaluateReport( + scenario: NativeAgentEvalScenario, + report: Omit, + providerInputs: ChatMessage[][] +): string[] { + const failures: string[] = [] + const expected = scenario.expected + + addMismatch(failures, 'status', report.outcome.status, expected.status) + addMismatch(failures, 'stopReason', report.outcome.stopReason, expected.stopReason) + addMismatch(failures, 'persistedStatus', report.persistedStatus, expected.persistedStatus) + addMismatch(failures, 'persistedRunId', report.persistedRunId, `request-${scenario.id}`) + addMismatch( + failures, + 'persistedRunOutcome', + report.persistedRunOutcome, + expected.persistedRunOutcome + ) + addMismatch( + failures, + 'persistedRunStopReason', + report.persistedRunStopReason, + expected.persistedRunStopReason + ) + addMismatch(failures, 'providerRounds', report.providerRounds, expected.providerRounds) + addMismatch(failures, 'toolCalls', report.toolCalls.total, expected.toolCalls) + addMismatch( + failures, + 'persistedProviderRounds', + report.persistedProviderRounds, + expected.providerRounds + ) + addMismatch(failures, 'persistedToolCalls', report.persistedToolCalls, expected.toolCalls) + + if (expected.finalTextIncludes && !report.finalText.includes(expected.finalTextIncludes)) { + failures.push(`finalText does not include ${expected.finalTextIncludes}`) + } + if ( + expected.terminalErrorIncludes && + !report.outcome.terminalError?.includes(expected.terminalErrorIncludes) + ) { + failures.push(`terminalError does not include ${expected.terminalErrorIncludes}`) + } + if ( + expected.errorMessageIncludes && + !report.outcome.errorMessage?.includes(expected.errorMessageIncludes) + ) { + failures.push(`errorMessage does not include ${expected.errorMessageIncludes}`) + } + + const toolMessages = providerInputs + .flat() + .filter((message) => message.role === 'tool') + .map(stringifyMessageContent) + for (const expectedContent of expected.toolMessageIncludes ?? []) { + if (!toolMessages.some((content) => content.includes(expectedContent))) { + failures.push(`provider input does not include tool message ${expectedContent}`) + } + } + + if (typeof expected.failedToolCalls === 'number') { + addMismatch(failures, 'failedToolCalls', report.toolCalls.failed, expected.failedToolCalls) + } + if (typeof expected.permissionRequests === 'number') { + addMismatch( + failures, + 'permissionRequests', + report.permissionRequests, + expected.permissionRequests + ) + } + for (const field of [ + 'inputTokens', + 'outputTokens', + 'totalTokens', + 'cachedInputTokens', + 'cacheWriteInputTokens' + ] as const) { + addMismatch(failures, field, report.usage[field], expected.usage[field]) + addMismatch( + failures, + `persistedUsage.${field}`, + report.persistedUsage[field], + expected.usage[field] + ) + } + + return failures +} + +export async function runNativeAgentEvalScenario( + scenario: NativeAgentEvalScenario +): Promise { + const abortController = new AbortController() + const { messageStore, state: messageState } = createMessageStore() + const toolHarness = createToolPresenter(scenario.tools ?? {}) + const providerInputs: ChatMessage[][] = [] + let providerRounds = 0 + let permissionRequests = 0 + + const coreStream = vi.fn(async function* (messages: ChatMessage[]) { + const roundIndex = providerRounds + providerRounds += 1 + providerInputs.push(clone(messages)) + const round = scenario.rounds[roundIndex] + if (!round) { + throw new Error(`No scripted provider round ${roundIndex + 1} for ${scenario.id}`) + } + + for (let eventIndex = 0; eventIndex < round.events.length; eventIndex += 1) { + if (round.abortBeforeEventIndex === eventIndex) { + abortController.abort() + } + yield round.events[eventIndex] + } + }) as unknown as ProcessParams['coreStream'] + + const tools = Object.keys(scenario.tools ?? {}).map(makeToolDefinition) + const toolOutputGuard = new ToolOutputGuard() + const sessionId = toAppSessionId(`eval-${scenario.id}`) + const messageId = `message-${scenario.id}` + const runId = `request-${scenario.id}` + const params: ProcessParams = { + run: createLoopRun({ + runId, + sessionId, + messageId, + abortController, + messages: [{ role: 'user', content: `Eval scenario: ${scenario.id}` }], + streamState: createState(), + resources: { toolDefinitions: tools, activeSkillNames: [] } + }), + toolCatalog: { + resolve: async () => tools + }, + toolExecution: createToolExecutionPort(toolHarness.presenter), + toolResults: createToolResultPort({ + outputGuard: toolOutputGuard, + normalize: async ({ content }) => content + }), + coreStream, + providerId: 'native-agent-eval', + modelId: 'scripted-provider', + modelConfig: { + maxTokens: 4096, + contextLength: 32_768, + vision: false, + functionCall: true, + reasoning: false, + type: ModelType.Chat + }, + temperature: 0, + maxTokens: 4096, + interleavedReasoning: DEFAULT_INTERLEAVED_REASONING, + permissionMode: scenario.permissionMode ?? 'full_access', + maxProviderRounds: scenario.maxProviderRounds, + shouldYieldForPendingInput: () => scenario.yieldForPendingInput === true, + notificationObserver: { + notify: (notification) => { + if (notification.event === 'PermissionRequest') { + permissionRequests += 1 + } + } + }, + io: { + messageStore, + tapeRecorder: { + appendToolFact: async () => ({ sessionId, entryId: 1 }) + } + } + } + + const startedAt = Date.now() + const result = await processStream(params) + const elapsedMs = Math.max(0, Date.now() - startedAt) + const toolCalls = { + total: toolHarness.calls.length, + succeeded: toolHarness.calls.filter((call) => call.status === 'success').length, + failed: toolHarness.calls.filter((call) => call.status === 'error').length + } + const baseReport: Omit = { + schemaVersion: 1, + scenarioId: scenario.id, + outcome: { + status: result.status, + stopReason: result.stopReason ?? null, + terminalError: result.terminalError ?? null, + errorMessage: result.errorMessage ?? null + }, + persistedStatus: messageState.persistedStatus, + persistedRunId: + typeof messageState.metadata.runId === 'string' ? messageState.metadata.runId : null, + persistedRunOutcome: + typeof messageState.metadata.runOutcome === 'string' + ? messageState.metadata.runOutcome + : null, + persistedRunStopReason: + typeof messageState.metadata.runStopReason === 'string' + ? messageState.metadata.runStopReason + : null, + persistedProviderRounds: normalizePersistedNumber(messageState.metadata, 'providerRounds'), + persistedToolCalls: normalizePersistedNumber(messageState.metadata, 'toolCalls'), + persistedUsage: normalizePersistedUsage(messageState.metadata), + providerRounds, + toolCalls, + permissionRequests, + elapsedMs, + usage: normalizeUsage(result), + finalText: collectFinalText(messageState.blocks) + } + const expectationFailures = evaluateReport(scenario, baseReport, providerInputs) + + return { + ...baseReport, + passed: expectationFailures.length === 0, + expectationFailures + } +} + +export function aggregateNativeAgentEvalReports( + scenarios: NativeAgentEvalScenario[], + reports: NativeAgentEvalReport[] +): NativeAgentEvalAggregate { + const scenarioById = new Map(scenarios.map((scenario) => [scenario.id, scenario])) + const providerRoundBudget = scenarios.reduce( + (total, scenario) => total + scenario.budget.maxProviderRounds, + 0 + ) + const toolCallBudget = scenarios.reduce( + (total, scenario) => total + scenario.budget.maxToolCalls, + 0 + ) + const totalProviderRounds = reports.reduce((total, report) => total + report.providerRounds, 0) + const totalToolCalls = reports.reduce((total, report) => total + report.toolCalls.total, 0) + const passed = reports.filter((report) => report.passed).length + + return { + schemaVersion: 1, + scenarios: reports.length, + passed, + passRate: reports.length === 0 ? 0 : passed / reports.length, + totalProviderRounds, + providerRoundBudget, + totalToolCalls, + toolCallBudget, + totalTokens: reports.reduce((total, report) => total + (report.usage.totalTokens ?? 0), 0), + withinCallBudgets: reports.every((report) => { + const scenario = scenarioById.get(report.scenarioId) + return ( + scenario !== undefined && + report.providerRounds <= scenario.budget.maxProviderRounds && + report.toolCalls.total <= scenario.budget.maxToolCalls + ) + }) + } +} + +export function completionRound( + text: string, + usage?: ScriptedProviderUsage +): ScriptedProviderRound { + return { + events: [ + { type: 'text', content: text }, + ...(usage + ? [ + { + type: 'usage' as const, + usage: { + prompt_tokens: usage.inputTokens, + completion_tokens: usage.outputTokens, + total_tokens: usage.totalTokens, + ...(usage.cachedInputTokens === undefined + ? {} + : { cached_tokens: usage.cachedInputTokens }), + ...(usage.cacheWriteInputTokens === undefined + ? {} + : { cache_write_tokens: usage.cacheWriteInputTokens }) + } + } + ] + : []), + { type: 'stop', stop_reason: 'complete' } + ] + } +} + +export function toolRound( + id: string, + name: string, + args: string, + usage?: ScriptedProviderUsage +): ScriptedProviderRound { + return { + events: [ + ...(usage + ? [ + { + type: 'usage' as const, + usage: { + prompt_tokens: usage.inputTokens, + completion_tokens: usage.outputTokens, + total_tokens: usage.totalTokens, + ...(usage.cachedInputTokens === undefined + ? {} + : { cached_tokens: usage.cachedInputTokens }), + ...(usage.cacheWriteInputTokens === undefined + ? {} + : { cache_write_tokens: usage.cacheWriteInputTokens }) + } + } + ] + : []), + { type: 'tool_call_start', tool_call_id: id, tool_call_name: name }, + { + type: 'tool_call_end', + tool_call_id: id, + tool_call_arguments_complete: args + }, + { type: 'stop', stop_reason: 'tool_use' } + ] + } +} diff --git a/test/main/evals/nativeAgent/nativeAgentBehavior.eval.test.ts b/test/main/evals/nativeAgent/nativeAgentBehavior.eval.test.ts new file mode 100644 index 000000000..50b4aedf0 --- /dev/null +++ b/test/main/evals/nativeAgent/nativeAgentBehavior.eval.test.ts @@ -0,0 +1,118 @@ +import { afterAll, afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { + aggregateNativeAgentEvalReports, + runNativeAgentEvalScenario, + type NativeAgentEvalReport +} from './harness' +import { NATIVE_AGENT_EVAL_SCENARIOS } from './scenarios' + +const EXPECTED_SCENARIO_IDS = [ + 'direct-completion', + 'max-tokens', + 'single-tool-round', + 'multiple-tool-rounds', + 'tool-failure-recovery', + 'permission-pause', + 'cancellation', + 'pending-input-yield', + 'max-provider-rounds', + 'max-tool-calls', + 'repeated-tool-no-progress', + 'empty-provider-output', + 'generic-provider-error', + 'context-window-error' +] + +describe('native Agent deterministic behavior eval', () => { + const reports: NativeAgentEvalReport[] = [] + + beforeEach(() => { + vi.useFakeTimers() + vi.setSystemTime(new Date('2026-01-01T00:00:00.000Z')) + }) + + afterEach(() => { + vi.useRealTimers() + }) + + it.each(NATIVE_AGENT_EVAL_SCENARIOS)('$id', async (scenario) => { + const report = await runNativeAgentEvalScenario(scenario) + reports.push(report) + + expect(report).toMatchObject({ + schemaVersion: 1, + scenarioId: scenario.id, + passed: true, + persistedRunId: `request-${scenario.id}`, + persistedRunOutcome: scenario.expected.persistedRunOutcome, + persistedRunStopReason: scenario.expected.persistedRunStopReason, + persistedProviderRounds: scenario.expected.providerRounds, + persistedToolCalls: scenario.expected.toolCalls, + persistedUsage: scenario.expected.usage, + providerRounds: expect.any(Number), + permissionRequests: expect.any(Number), + elapsedMs: expect.any(Number), + usage: { + inputTokens: expect.toSatisfy((value) => value === null || typeof value === 'number'), + outputTokens: expect.toSatisfy((value) => value === null || typeof value === 'number'), + totalTokens: expect.toSatisfy((value) => value === null || typeof value === 'number'), + cachedInputTokens: expect.toSatisfy((value) => value === null || typeof value === 'number'), + cacheWriteInputTokens: expect.toSatisfy( + (value) => value === null || typeof value === 'number' + ) + } + }) + expect(report.expectationFailures, JSON.stringify(report, null, 2)).toEqual([]) + expect(report.providerRounds).toBe(scenario.expected.providerRounds) + expect(report.toolCalls.total).toBe(scenario.expected.toolCalls) + expect(report.providerRounds).toBeLessThanOrEqual(scenario.budget.maxProviderRounds) + expect(report.toolCalls.total).toBeLessThanOrEqual(scenario.budget.maxToolCalls) + expect(report.elapsedMs).toBeGreaterThanOrEqual(0) + }) + + afterAll(() => { + expect(NATIVE_AGENT_EVAL_SCENARIOS.map((scenario) => scenario.id)).toEqual( + EXPECTED_SCENARIO_IDS + ) + expect(reports.map((report) => report.scenarioId).sort()).toEqual( + [...EXPECTED_SCENARIO_IDS].sort() + ) + + const aggregate = aggregateNativeAgentEvalReports(NATIVE_AGENT_EVAL_SCENARIOS, reports) + const expectedTotalTokens = NATIVE_AGENT_EVAL_SCENARIOS.reduce( + (total, scenario) => total + (scenario.expected.usage.totalTokens ?? 0), + 0 + ) + const expectedProviderRounds = NATIVE_AGENT_EVAL_SCENARIOS.reduce( + (total, scenario) => total + scenario.expected.providerRounds, + 0 + ) + const expectedToolCalls = NATIVE_AGENT_EVAL_SCENARIOS.reduce( + (total, scenario) => total + scenario.expected.toolCalls, + 0 + ) + + expect(aggregate).toEqual({ + schemaVersion: 1, + scenarios: EXPECTED_SCENARIO_IDS.length, + passed: EXPECTED_SCENARIO_IDS.length, + passRate: 1, + totalProviderRounds: expectedProviderRounds, + providerRoundBudget: 150, + totalToolCalls: expectedToolCalls, + toolCallBudget: 139, + totalTokens: expectedTotalTokens, + withinCallBudgets: true + }) + + const overBudgetReports = reports.map((report) => + report.scenarioId === 'multiple-tool-rounds' + ? { ...report, providerRounds: 5, toolCalls: { ...report.toolCalls, total: 4 } } + : report + ) + expect( + aggregateNativeAgentEvalReports(NATIVE_AGENT_EVAL_SCENARIOS, overBudgetReports) + .withinCallBudgets + ).toBe(false) + }) +}) diff --git a/test/main/evals/nativeAgent/scenarios.ts b/test/main/evals/nativeAgent/scenarios.ts new file mode 100644 index 000000000..5aa50d628 --- /dev/null +++ b/test/main/evals/nativeAgent/scenarios.ts @@ -0,0 +1,446 @@ +import type { + NativeAgentEvalScenario, + NativeAgentEvalUsage, + ScriptedProviderRound, + ScriptedProviderUsage +} from './harness' +import { completionRound, toolRound } from './harness' + +const DIRECT_USAGE = { inputTokens: 10, outputTokens: 4, totalTokens: 14 } +const MAX_TOKENS_USAGE = { inputTokens: 11, outputTokens: 4, totalTokens: 15 } +const SINGLE_TOOL_USAGE = { inputTokens: 18, outputTokens: 6, totalTokens: 24 } +const MULTI_TOOL_ROUND_USAGES = [ + { + inputTokens: 7, + outputTokens: 1, + totalTokens: 8, + cachedInputTokens: 2, + cacheWriteInputTokens: 1 + }, + { inputTokens: 8, outputTokens: 2, totalTokens: 10, cachedInputTokens: 3 }, + { inputTokens: 10, outputTokens: 5, totalTokens: 15, cacheWriteInputTokens: 2 } +] satisfies ScriptedProviderUsage[] +const MULTI_TOOL_USAGE = { + inputTokens: 25, + outputTokens: 8, + totalTokens: 33, + cachedInputTokens: 5, + cacheWriteInputTokens: 3 +} +const TOOL_FAILURE_USAGE = { inputTokens: 12, outputTokens: 5, totalTokens: 17 } +const GENERIC_ERROR_USAGE = { inputTokens: 8, outputTokens: 1, totalTokens: 9 } +const CONTEXT_WINDOW_USAGE = { inputTokens: 20, outputTokens: 1, totalTokens: 21 } +const MAX_TOOL_CALLS_USAGE = { inputTokens: 130, outputTokens: 1, totalTokens: 131 } +const MAX_TOOL_CALLS = 128 + +function expectedUsage(usage?: ScriptedProviderUsage): NativeAgentEvalUsage { + return { + inputTokens: usage?.inputTokens ?? null, + outputTokens: usage?.outputTokens ?? null, + totalTokens: usage?.totalTokens ?? null, + cachedInputTokens: usage?.cachedInputTokens ?? null, + cacheWriteInputTokens: usage?.cacheWriteInputTokens ?? null + } +} + +function maxToolCallRounds(): ScriptedProviderRound[] { + return Array.from({ length: MAX_TOOL_CALLS + 1 }, (_, index) => { + const iteration = index + 1 + const events: ScriptedProviderRound['events'] = [] + if (index === MAX_TOOL_CALLS) { + events.push({ + type: 'usage', + usage: { + prompt_tokens: MAX_TOOL_CALLS_USAGE.inputTokens, + completion_tokens: MAX_TOOL_CALLS_USAGE.outputTokens, + total_tokens: MAX_TOOL_CALLS_USAGE.totalTokens + } + }) + } + events.push( + { + type: 'tool_call_start', + tool_call_id: `call-action-${iteration}`, + tool_call_name: 'action' + }, + { + type: 'tool_call_end', + tool_call_id: `call-action-${iteration}`, + tool_call_arguments_complete: JSON.stringify({ iteration }) + }, + { type: 'stop', stop_reason: 'tool_use' } + ) + return { events } + }) +} + +export const NATIVE_AGENT_EVAL_SCENARIOS: NativeAgentEvalScenario[] = [ + { + id: 'direct-completion', + rounds: [completionRound('Direct answer', DIRECT_USAGE)], + budget: { maxProviderRounds: 1, maxToolCalls: 0 }, + expected: { + status: 'completed', + stopReason: 'complete', + persistedStatus: 'sent', + persistedRunOutcome: 'completed', + persistedRunStopReason: 'complete', + providerRounds: 1, + toolCalls: 0, + finalTextIncludes: 'Direct answer', + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage(DIRECT_USAGE) + } + }, + { + id: 'max-tokens', + rounds: [ + { + events: [ + { + type: 'usage', + usage: { + prompt_tokens: MAX_TOKENS_USAGE.inputTokens, + completion_tokens: MAX_TOKENS_USAGE.outputTokens, + total_tokens: MAX_TOKENS_USAGE.totalTokens + } + }, + { type: 'text', content: 'Truncated answer' }, + { type: 'stop', stop_reason: 'max_tokens' } + ] + } + ], + budget: { maxProviderRounds: 1, maxToolCalls: 0 }, + expected: { + status: 'completed', + stopReason: 'max_tokens', + persistedStatus: 'sent', + persistedRunOutcome: 'completed', + persistedRunStopReason: 'max_tokens', + providerRounds: 1, + toolCalls: 0, + finalTextIncludes: 'Truncated answer', + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage(MAX_TOKENS_USAGE) + } + }, + { + id: 'single-tool-round', + rounds: [ + toolRound('call-read-1', 'read', '{"path":"src/main.ts"}'), + completionRound('Read completed', SINGLE_TOOL_USAGE) + ], + tools: { + read: { response: 'export const answer = 42' } + }, + budget: { maxProviderRounds: 2, maxToolCalls: 1 }, + expected: { + status: 'completed', + stopReason: 'complete', + persistedStatus: 'sent', + persistedRunOutcome: 'completed', + persistedRunStopReason: 'complete', + providerRounds: 2, + toolCalls: 1, + finalTextIncludes: 'Read completed', + toolMessageIncludes: ['export const answer = 42'], + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage(SINGLE_TOOL_USAGE) + } + }, + { + id: 'multiple-tool-rounds', + rounds: [ + toolRound('call-search-1', 'grep', '{"query":"answer"}', MULTI_TOOL_ROUND_USAGES[0]), + toolRound('call-read-2', 'read', '{"path":"src/answer.ts"}', MULTI_TOOL_ROUND_USAGES[1]), + completionRound('Located the answer', MULTI_TOOL_ROUND_USAGES[2]) + ], + tools: { + grep: { response: '[{"path":"src/answer.ts","lineNumber":1}]' }, + read: { response: 'export const answer = 42' } + }, + budget: { maxProviderRounds: 4, maxToolCalls: 3 }, + expected: { + status: 'completed', + stopReason: 'complete', + persistedStatus: 'sent', + persistedRunOutcome: 'completed', + persistedRunStopReason: 'complete', + providerRounds: 3, + toolCalls: 2, + finalTextIncludes: 'Located the answer', + toolMessageIncludes: ['src/answer.ts', 'export const answer = 42'], + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage(MULTI_TOOL_USAGE) + } + }, + { + id: 'tool-failure-recovery', + rounds: [ + toolRound('call-lookup-1', 'lookup', '{"id":"missing"}'), + completionRound('Recovered after tool failure', TOOL_FAILURE_USAGE) + ], + tools: { + lookup: { error: 'simulated lookup failure' } + }, + budget: { maxProviderRounds: 2, maxToolCalls: 1 }, + expected: { + status: 'completed', + stopReason: 'complete', + persistedStatus: 'sent', + persistedRunOutcome: 'completed', + persistedRunStopReason: 'complete', + providerRounds: 2, + toolCalls: 1, + finalTextIncludes: 'Recovered after tool failure', + toolMessageIncludes: ['Error: simulated lookup failure'], + failedToolCalls: 1, + permissionRequests: 0, + usage: expectedUsage(TOOL_FAILURE_USAGE) + } + }, + { + id: 'permission-pause', + rounds: [toolRound('call-write-1', 'write', '{"path":"result.txt","content":"ok"}')], + tools: { + write: { + response: 'written', + permission: { + permissionType: 'write', + description: 'Allow writing result.txt' + } + } + }, + permissionMode: 'ask_user', + budget: { maxProviderRounds: 1, maxToolCalls: 0 }, + expected: { + status: 'paused', + stopReason: null, + persistedStatus: 'pending', + persistedRunOutcome: 'paused', + persistedRunStopReason: 'interaction', + providerRounds: 1, + toolCalls: 0, + failedToolCalls: 0, + permissionRequests: 1, + usage: expectedUsage() + } + }, + { + id: 'cancellation', + rounds: [ + { + events: [ + { type: 'text', content: 'Partial answer' }, + { type: 'stop', stop_reason: 'complete' } + ], + abortBeforeEventIndex: 1 + } + ], + budget: { maxProviderRounds: 1, maxToolCalls: 0 }, + expected: { + status: 'aborted', + stopReason: 'user_stop', + persistedStatus: 'error', + persistedRunOutcome: 'aborted', + persistedRunStopReason: 'user_stop', + providerRounds: 1, + toolCalls: 0, + finalTextIncludes: 'common.error.userCanceledGeneration', + errorMessageIncludes: 'common.error.userCanceledGeneration', + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage() + } + }, + { + id: 'pending-input-yield', + rounds: [toolRound('call-read-pending', 'read', '{"path":"pending.txt"}')], + tools: { + read: { response: 'pending result' } + }, + yieldForPendingInput: true, + budget: { maxProviderRounds: 1, maxToolCalls: 1 }, + expected: { + status: 'completed', + stopReason: 'pending_input', + persistedStatus: 'sent', + persistedRunOutcome: 'completed', + persistedRunStopReason: 'pending_input', + providerRounds: 1, + toolCalls: 1, + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage() + } + }, + { + id: 'max-provider-rounds', + rounds: [toolRound('call-loop-1', 'loop', '{}')], + tools: { + loop: { response: 'continue' } + }, + maxProviderRounds: 1, + budget: { maxProviderRounds: 1, maxToolCalls: 1 }, + expected: { + status: 'error', + stopReason: 'max_turns', + persistedStatus: 'error', + persistedRunOutcome: 'error', + persistedRunStopReason: 'max_turns', + providerRounds: 1, + toolCalls: 1, + terminalErrorIncludes: 'Maximum agent turns exceeded (1).', + errorMessageIncludes: 'Maximum agent turns exceeded (1).', + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage() + } + }, + { + id: 'max-tool-calls', + rounds: maxToolCallRounds(), + tools: { + action: { response: 'completed action' } + }, + budget: { maxProviderRounds: 129, maxToolCalls: 128 }, + expected: { + status: 'completed', + stopReason: 'max_tool_calls', + persistedStatus: 'sent', + persistedRunOutcome: 'completed', + persistedRunStopReason: 'max_tool_calls', + providerRounds: 129, + toolCalls: 128, + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage(MAX_TOOL_CALLS_USAGE) + } + }, + { + id: 'repeated-tool-no-progress', + rounds: [ + toolRound('call-loop-1', 'loop', '{"target":"same"}'), + toolRound('call-loop-2', 'loop', '{"target":"same"}'), + toolRound('call-loop-3', 'loop', '{"target":"same"}'), + toolRound('call-loop-4', 'loop', '{"target":"same"}') + ], + tools: { + loop: { response: 'unchanged result' } + }, + budget: { maxProviderRounds: 4, maxToolCalls: 4 }, + expected: { + status: 'error', + stopReason: 'no_progress', + persistedStatus: 'error', + persistedRunOutcome: 'error', + persistedRunStopReason: 'no_progress', + providerRounds: 4, + toolCalls: 4, + finalTextIncludes: 'four identical tool batches produced no progress', + terminalErrorIncludes: 'four identical tool batches produced no progress', + errorMessageIncludes: 'four identical tool batches produced no progress', + toolMessageIncludes: ['agent_no_progress'], + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage() + } + }, + { + id: 'empty-provider-output', + rounds: [{ events: [{ type: 'stop', stop_reason: 'complete' }] }], + budget: { maxProviderRounds: 1, maxToolCalls: 0 }, + expected: { + status: 'error', + stopReason: 'empty_response', + persistedStatus: 'error', + persistedRunOutcome: 'error', + persistedRunStopReason: 'empty_response', + providerRounds: 1, + toolCalls: 0, + finalTextIncludes: 'common.error.noModelResponse', + terminalErrorIncludes: 'common.error.noModelResponse', + errorMessageIncludes: 'common.error.noModelResponse', + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage() + } + }, + { + id: 'generic-provider-error', + rounds: [ + { + events: [ + { + type: 'usage', + usage: { + prompt_tokens: GENERIC_ERROR_USAGE.inputTokens, + completion_tokens: GENERIC_ERROR_USAGE.outputTokens, + total_tokens: GENERIC_ERROR_USAGE.totalTokens + } + }, + { type: 'text', content: 'Partial provider answer' }, + { type: 'error', error_message: 'Rate limit exceeded' }, + { type: 'stop', stop_reason: 'complete' } + ] + } + ], + budget: { maxProviderRounds: 1, maxToolCalls: 0 }, + expected: { + status: 'error', + stopReason: 'provider_error', + persistedStatus: 'error', + persistedRunOutcome: 'error', + persistedRunStopReason: 'provider_error', + providerRounds: 1, + toolCalls: 0, + finalTextIncludes: 'Rate limit exceeded', + terminalErrorIncludes: 'Rate limit exceeded', + errorMessageIncludes: 'Rate limit exceeded', + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage(GENERIC_ERROR_USAGE) + } + }, + { + id: 'context-window-error', + rounds: [ + { + events: [ + { + type: 'usage', + usage: { + prompt_tokens: CONTEXT_WINDOW_USAGE.inputTokens, + completion_tokens: CONTEXT_WINDOW_USAGE.outputTokens, + total_tokens: CONTEXT_WINDOW_USAGE.totalTokens + } + }, + { + type: 'error', + error_message: 'maximum context length exceeded' + } + ] + } + ], + budget: { maxProviderRounds: 1, maxToolCalls: 0 }, + expected: { + status: 'error', + stopReason: 'context_window', + persistedStatus: 'error', + persistedRunOutcome: 'error', + persistedRunStopReason: 'context_window', + providerRounds: 1, + toolCalls: 0, + finalTextIncludes: 'maximum context length exceeded', + terminalErrorIncludes: 'maximum context length exceeded', + errorMessageIncludes: 'maximum context length exceeded', + failedToolCalls: 0, + permissionRequests: 0, + usage: expectedUsage(CONTEXT_WINDOW_USAGE) + } + } +] diff --git a/test/main/lib/awaitWithAbort.test.ts b/test/main/lib/awaitWithAbort.test.ts new file mode 100644 index 000000000..87bfb06f9 --- /dev/null +++ b/test/main/lib/awaitWithAbort.test.ts @@ -0,0 +1,72 @@ +import { describe, expect, it, vi } from 'vitest' + +import { awaitWithAbort } from '@/lib/awaitWithAbort' + +function deferred() { + let resolve!: (value: T) => void + let reject!: (reason?: unknown) => void + const promise = new Promise((promiseResolve, promiseReject) => { + resolve = promiseResolve + reject = promiseReject + }) + return { promise, resolve, reject } +} + +async function flushUnhandledRejections(): Promise { + await new Promise((resolve) => setImmediate(resolve)) + await new Promise((resolve) => setImmediate(resolve)) +} + +describe('awaitWithAbort', () => { + it('consumes a late source rejection when the signal was aborted before the wait began', async () => { + const source = deferred() + const controller = new AbortController() + const lateError = new Error('late source failure') + const unhandled = vi.fn() + controller.abort() + + await expect(awaitWithAbort(source.promise, controller.signal)).rejects.toMatchObject({ + name: 'AbortError' + }) + + process.on('unhandledRejection', unhandled) + try { + source.reject(lateError) + await flushUnhandledRejections() + expect(unhandled.mock.calls.some(([reason]) => reason === lateError)).toBe(false) + } finally { + process.off('unhandledRejection', unhandled) + } + }) + + it('consumes a late source rejection after an in-flight abort wins', async () => { + const source = deferred() + const controller = new AbortController() + const lateError = new Error('late source failure') + const unhandled = vi.fn() + const waiting = awaitWithAbort(source.promise, controller.signal) + + controller.abort() + await expect(waiting).rejects.toMatchObject({ name: 'AbortError' }) + + process.on('unhandledRejection', unhandled) + try { + source.reject(lateError) + await flushUnhandledRejections() + expect(unhandled.mock.calls.some(([reason]) => reason === lateError)).toBe(false) + } finally { + process.off('unhandledRejection', unhandled) + } + }) + + it('still propagates a source rejection while the signal is active', async () => { + const source = deferred() + const controller = new AbortController() + const sourceError = new Error('source failure') + const waiting = awaitWithAbort(source.promise, controller.signal) + + source.reject(sourceError) + + await expect(waiting).rejects.toBe(sourceError) + }) +}) diff --git a/test/main/lib/toolCallImagePreviews.test.ts b/test/main/lib/toolCallImagePreviews.test.ts index 052157352..fbb89cb06 100644 --- a/test/main/lib/toolCallImagePreviews.test.ts +++ b/test/main/lib/toolCallImagePreviews.test.ts @@ -121,4 +121,67 @@ describe('extractToolCallImagePreviews', () => { } ]) }) + + it('does not start image caching when already cancelled', async () => { + const cacheImage = vi.fn(async () => 'imgcache://should-not-run.png') + const abortController = new AbortController() + abortController.abort() + + await expect( + extractToolCallImagePreviews({ + content: [{ type: 'image', data: 'AAAA', mimeType: 'image/png' }], + cacheImage, + signal: abortController.signal + }) + ).rejects.toMatchObject({ name: 'AbortError' }) + expect(cacheImage).not.toHaveBeenCalled() + }) + + it('rejects promptly when cancellation lands during image caching', async () => { + const cacheImage = vi.fn(() => new Promise(() => {})) + const abortController = new AbortController() + + const extracting = extractToolCallImagePreviews({ + content: [{ type: 'image', data: 'AAAA', mimeType: 'image/png' }], + cacheImage, + signal: abortController.signal + }) + await vi.waitFor(() => expect(cacheImage).toHaveBeenCalledOnce()) + + abortController.abort() + + await expect(extracting).rejects.toMatchObject({ name: 'AbortError' }) + }) + + it('observes a late cache failure when caching synchronously cancels', async () => { + let rejectCache!: (reason?: unknown) => void + const cachePromise = new Promise((_, reject) => { + rejectCache = reject + }) + const abortController = new AbortController() + const lateError = new Error('late cache failure') + const unhandled = vi.fn() + const cacheImage = vi.fn(() => { + abortController.abort() + return cachePromise + }) + + await expect( + extractToolCallImagePreviews({ + content: [{ type: 'image', data: 'AAAA', mimeType: 'image/png' }], + cacheImage, + signal: abortController.signal + }) + ).rejects.toMatchObject({ name: 'AbortError' }) + + process.on('unhandledRejection', unhandled) + try { + rejectCache(lateError) + await new Promise((resolve) => setImmediate(resolve)) + await new Promise((resolve) => setImmediate(resolve)) + expect(unhandled.mock.calls.some(([reason]) => reason === lateError)).toBe(false) + } finally { + process.off('unhandledRejection', unhandled) + } + }) }) diff --git a/test/main/presenter/agentRuntimePresenter/accumulator.test.ts b/test/main/presenter/agentRuntimePresenter/accumulator.test.ts index 461d554d1..906c99515 100644 --- a/test/main/presenter/agentRuntimePresenter/accumulator.test.ts +++ b/test/main/presenter/agentRuntimePresenter/accumulator.test.ts @@ -1,5 +1,5 @@ import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest' -import { accumulate } from '@/presenter/agentRuntimePresenter/accumulator' +import { accumulate, commitRoundUsage } from '@/presenter/agentRuntimePresenter/accumulator' import { createState } from '@/presenter/agentRuntimePresenter/types' import type { StreamState } from '@/presenter/agentRuntimePresenter/types' @@ -266,6 +266,14 @@ describe('accumulate', () => { } }) + expect(state.roundUsage).toEqual({ + inputTokens: 10, + outputTokens: 5, + totalTokens: 15, + cachedInputTokens: 3, + cacheWriteInputTokens: 2 + }) + commitRoundUsage(state) expect(state.metadata.inputTokens).toBe(10) expect(state.metadata.outputTokens).toBe(5) expect(state.metadata.totalTokens).toBe(15) diff --git a/test/main/presenter/agentRuntimePresenter/acpCompatibilityAdapters.test.ts b/test/main/presenter/agentRuntimePresenter/acpCompatibilityAdapters.test.ts index 42c48251b..a751de65b 100644 --- a/test/main/presenter/agentRuntimePresenter/acpCompatibilityAdapters.test.ts +++ b/test/main/presenter/agentRuntimePresenter/acpCompatibilityAdapters.test.ts @@ -231,7 +231,10 @@ describe('ACP compatibility adapters', () => { const blocks = JSON.parse(message?.content ?? '[]') const tapeRecord = harness.getAssistantTapeRecord() - expect(settlement).toEqual({ status: 'completed', stopReason: 'complete' }) + expect(settlement).toEqual({ + status: 'completed', + stopReason: 'complete' + }) expect(message?.status).toBe('sent') expect(blocks).toEqual( expect.arrayContaining([ @@ -332,19 +335,26 @@ describe('ACP compatibility adapters', () => { const blocks = JSON.parse(message?.content ?? '[]') const tapeRecord = harness.getAssistantTapeRecord() - expect(settlement).toEqual({ status: 'completed', stopReason: 'complete' }) - expect(message?.status).toBe('sent') + expect(settlement).toEqual({ + status: 'error', + stopReason: 'error', + errorMessage: 'ACP: prompt failed' + }) + expect(message?.status).toBe('error') expect(blocks).toEqual([ expect.objectContaining({ type: 'error', content: 'ACP: prompt failed', status: 'error' }) ]) - expect(tapeRecord?.status).toBe('sent') + expect(tapeRecord?.status).toBe('error') expect(JSON.parse(tapeRecord?.content ?? '[]')).toEqual(blocks) expect(publishDeepchatEvent).toHaveBeenCalledWith( - 'chat.stream.completed', - expect.objectContaining({ messageId: harness.handle.messageId }) + 'chat.stream.failed', + expect.objectContaining({ + messageId: harness.handle.messageId, + error: 'ACP: prompt failed' + }) ) expect(publishDeepchatEvent).not.toHaveBeenCalledWith( - 'chat.stream.failed', + 'chat.stream.completed', expect.objectContaining({ messageId: harness.handle.messageId }) ) }) diff --git a/test/main/presenter/agentRuntimePresenter/agentRuntimePresenter.test.ts b/test/main/presenter/agentRuntimePresenter/agentRuntimePresenter.test.ts index 0df75ce95..cd3101cc4 100644 --- a/test/main/presenter/agentRuntimePresenter/agentRuntimePresenter.test.ts +++ b/test/main/presenter/agentRuntimePresenter/agentRuntimePresenter.test.ts @@ -246,6 +246,7 @@ function createMockSqlitePresenter() { const deepchatMessagesTable = { insert: vi.fn(), updateContent: vi.fn(), + updateMetadata: vi.fn(), updateStatus: vi.fn(), incrementOrderSeqFrom: vi.fn(), updateContentAndStatus: vi.fn(), @@ -1509,7 +1510,7 @@ describe('AgentRuntimePresenter', () => { 's1', 'openai', 'gpt-4', - 'full_access', + 'default', expect.objectContaining({ systemPrompt: 'You are a helpful assistant.', temperature: 0.7, @@ -1523,7 +1524,7 @@ describe('AgentRuntimePresenter', () => { status: 'idle', providerId: 'openai', modelId: 'gpt-4', - permissionMode: 'full_access' + permissionMode: 'default' }) }) @@ -1531,14 +1532,14 @@ describe('AgentRuntimePresenter', () => { await agent.initSession('s1', { providerId: 'openai', modelId: 'gpt-4', - permissionMode: 'default' + permissionMode: 'full_access' }) expect(sqlitePresenter.deepchatSessionsTable.create).toHaveBeenCalledWith( 's1', 'openai', 'gpt-4', - 'default', + 'full_access', expect.objectContaining({ systemPrompt: 'You are a helpful assistant.', temperature: 0.7, @@ -1548,7 +1549,7 @@ describe('AgentRuntimePresenter', () => { ) const state = await agent.getSessionState('s1') - expect(state?.permissionMode).toBe('default') + expect(state?.permissionMode).toBe('full_access') }) }) @@ -1576,6 +1577,25 @@ describe('AgentRuntimePresenter', () => { }) }) + it('projects pending interactions without caching a generating runtime state', async () => { + sqlitePresenter.deepchatSessionsTable.get.mockReturnValue({ + id: 's1', + provider_id: 'openai', + model_id: 'gpt-4', + permission_mode: 'full_access' + }) + const hasPendingInteractions = vi + .spyOn(agent as any, 'hasPendingInteractions') + .mockReturnValue(true) + + await expect(agent.getSessionState('s1')).resolves.toMatchObject({ status: 'generating' }) + expect(getRuntimeState(agent, 's1').status).toBe('idle') + + hasPendingInteractions.mockReturnValue(false) + await expect(agent.getSessionState('s1')).resolves.toMatchObject({ status: 'idle' }) + expect(getRuntimeState(agent, 's1').status).toBe('idle') + }) + it('returns null for unknown session', async () => { sqlitePresenter.deepchatSessionsTable.get.mockReturnValue(undefined) const state = await agent.getSessionState('unknown') @@ -4091,6 +4111,23 @@ describe('AgentRuntimePresenter', () => { ) }) + it('does not publish generation settings when persistence fails', async () => { + await agent.initSession('s1', { providerId: 'openai', modelId: 'gpt-4' }) + const instance = agent.deepChatRuntime.getOrHydrate(toAppSessionId('s1')) + const previousSettings = instance.getGenerationSettings() + const invalidateSystemPromptCache = vi.spyOn(instance, 'invalidateSystemPromptCache') + sqlitePresenter.deepchatSessionsTable.updateGenerationSettings.mockImplementationOnce(() => { + throw new Error('write failed') + }) + + await expect( + agent.updateGenerationSettings('s1', { systemPrompt: 'uncommitted prompt' }) + ).rejects.toThrow('write failed') + + expect(instance.getGenerationSettings()).toEqual(previousSettings) + expect(invalidateSystemPromptCache).not.toHaveBeenCalled() + }) + it('keeps image generation settings for OpenAI-compatible providers', async () => { configPresenter.getModelConfig.mockImplementation((modelId: string, providerId: string) => { if (providerId === 'aihubmix' && modelId === 'gpt-image-2') { @@ -4139,7 +4176,7 @@ describe('AgentRuntimePresenter', () => { 's1', 'aihubmix', 'gpt-image-2', - 'full_access', + 'default', expect.objectContaining({ imageGeneration: { size: '1024x1024', @@ -4244,7 +4281,7 @@ describe('AgentRuntimePresenter', () => { 's1', 'openai', 'gpt-4', - 'full_access', + 'default', expect.objectContaining({ forceInterleavedThinkingCompat: true }) @@ -4742,6 +4779,63 @@ describe('AgentRuntimePresenter', () => { }) }) + it('publishes a model switch only after its transaction commits', async () => { + const transaction = vi.fn((fn: () => unknown) => () => fn()) + sqlitePresenter.getDatabase.mockReturnValue({ transaction }) + await agent.initSession('s1', { + providerId: 'openai', + modelId: 'gpt-4', + permissionMode: 'full_access' + }) + const instance = agent.deepChatRuntime.getOrHydrate(toAppSessionId('s1')) + const previousState = { ...getRuntimeState(agent, 's1') } + const previousSettings = instance.getGenerationSettings() + const invalidateSystemPromptCache = vi.spyOn(instance, 'invalidateSystemPromptCache') + const invalidateToolProfileCache = vi.spyOn(instance, 'invalidateToolProfileCache') + sqlitePresenter.deepchatSessionsTable.updateGenerationSettings.mockImplementationOnce(() => { + throw new Error('write failed') + }) + + await expect(agent.setSessionModel('s1', 'anthropic', 'claude-3-5-sonnet')).rejects.toThrow( + 'write failed' + ) + + expect(transaction).toHaveBeenCalledOnce() + expect(getRuntimeState(agent, 's1')).toEqual(previousState) + expect(instance.getGenerationSettings()).toEqual(previousSettings) + expect(invalidateSystemPromptCache).not.toHaveBeenCalled() + expect(invalidateToolProfileCache).not.toHaveBeenCalled() + }) + + it('does not publish agent context when its transaction fails', async () => { + await agent.initSession('s1', { providerId: 'openai', modelId: 'gpt-4' }) + const instance = agent.deepChatRuntime.getOrHydrate(toAppSessionId('s1')) + const previousState = { ...getRuntimeState(agent, 's1') } + const previousSettings = instance.getGenerationSettings() + const invalidateSystemPromptCache = vi.spyOn(instance, 'invalidateSystemPromptCache') + const invalidateToolProfileCache = vi.spyOn(instance, 'invalidateToolProfileCache') + sqlitePresenter.deepchatSessionsTable.updateGenerationSettings.mockImplementationOnce(() => { + throw new Error('write failed') + }) + + await expect( + agent.setSessionAgentContext('s1', { + agentId: 'other-agent', + providerId: 'anthropic', + modelId: 'claude-3-5-sonnet', + projectDir: '/private/project', + permissionMode: 'full_access' + }) + ).rejects.toThrow('write failed') + + expect(getRuntimeState(agent, 's1')).toEqual(previousState) + expect(instance.getAgentId()).toBe('deepchat') + expect(instance.getProjectDir()).toBeNull() + expect(instance.getGenerationSettings()).toEqual(previousSettings) + expect(invalidateSystemPromptCache).not.toHaveBeenCalled() + expect(invalidateToolProfileCache).not.toHaveBeenCalled() + }) + it('drops unsupported reasoning and verbosity settings when switching models', async () => { configPresenter.getModelConfig.mockImplementation((modelId: string, providerId: string) => { if (providerId === 'openai' && modelId === 'gpt-4o-mini') { @@ -4859,12 +4953,181 @@ describe('AgentRuntimePresenter', () => { expect(state!.status).toBe('idle') }) - it('finalizes the active assistant message on stop', async () => { + it.each([ + { + interactionKind: 'question', + toolCallId: 'tc-question', + actionBlock: { + type: 'action', + action_type: 'question_request', + status: 'pending', + timestamp: 2, + content: 'Continue?', + tool_call: { id: 'tc-question', name: 'ask_question', params: '{}' }, + extra: { + needsUserAction: true, + questionText: 'Continue?', + questionOptions: [{ label: 'Yes' }] + } + } + }, + { + interactionKind: 'permission', + toolCallId: 'tc-permission', + actionBlock: { + type: 'action', + action_type: 'tool_call_permission', + status: 'pending', + timestamp: 2, + content: 'Allow file write?', + tool_call: { id: 'tc-permission', name: 'write_file', params: '{}' }, + extra: { + needsUserAction: true, + permissionType: 'write', + toolName: 'write_file', + serverName: 'agent-filesystem', + permissionRequest: JSON.stringify({ + permissionType: 'write', + description: 'Allow file write?', + toolName: 'write_file', + serverName: 'agent-filesystem', + paths: ['output.txt'] + }) + } + } + } + ])( + 'settles a paused $interactionKind interaction when generation is canceled', + async ({ interactionKind, toolCallId, actionBlock }) => { + await agent.initSession('s1', { providerId: 'openai', modelId: 'gpt-4' }) + const expectedUsage = { + inputTokens: 7, + outputTokens: 5, + totalTokens: 12, + cachedInputTokens: 2, + cacheWriteInputTokens: 1 + } + const accumulatedMetadata = { + runId: `paused-${interactionKind}-run`, + provider: 'openai', + model: 'gpt-4', + ...expectedUsage, + providerRounds: 3, + toolCalls: 4, + runOutcome: 'paused', + runStopReason: 'interaction' + } + const blocks: AssistantMessageBlock[] = [ + { + type: 'tool_call', + status: 'pending', + timestamp: 1, + tool_call: { + id: toolCallId, + name: actionBlock.tool_call.name, + params: '{}', + response: '' + } + }, + actionBlock as AssistantMessageBlock + ] + const row: any = { + id: `paused-${interactionKind}-message`, + session_id: 's1', + order_seq: 2, + role: 'assistant', + content: JSON.stringify(blocks), + status: 'pending', + is_context_edge: 0, + metadata: JSON.stringify(accumulatedMetadata), + created_at: 1, + updated_at: 1 + } + sqlitePresenter.deepchatMessagesTable.get.mockImplementation((id: string) => + id === row.id ? row : undefined + ) + sqlitePresenter.deepchatMessagesTable.getBySession.mockReturnValue([row]) + sqlitePresenter.deepchatMessagesTable.updateContentAndStatus.mockImplementation( + (id: string, content: string, status: string, metadata?: string) => { + if (id !== row.id) return + row.content = content + row.status = status + if (metadata !== undefined) row.metadata = metadata + } + ) + + const instance = agent.deepChatRuntime.getHydrated(toAppSessionId('s1')) + expect((agent as any).refreshPendingInteractionsFromStore('s1')).toBe(true) + expect(instance?.getPendingInteractions()).toHaveLength(1) + getRuntimeState(agent, 's1').status = 'generating' + + await agent.cancelGeneration('s1') + + const errorWrite = + sqlitePresenter.deepchatMessagesTable.updateContentAndStatus.mock.calls.find( + (call) => call[0] === row.id && call[2] === 'error' + ) + expect(errorWrite).toBeDefined() + const canceledBlocks = JSON.parse(errorWrite?.[1] ?? '[]') + expect(canceledBlocks.slice(0, 2)).toEqual([ + expect.objectContaining({ type: 'tool_call', status: 'error' }), + expect.objectContaining({ type: 'action', status: 'error' }) + ]) + expect(canceledBlocks.at(-1)).toEqual( + expect.objectContaining({ + type: 'error', + status: 'error', + content: 'common.error.userCanceledGeneration' + }) + ) + expect(JSON.parse(errorWrite?.[3] ?? '{}')).toMatchObject({ + ...accumulatedMetadata, + runOutcome: 'aborted', + runStopReason: 'user_stop' + }) + expect(instance?.getPendingInteractions()).toEqual([]) + expect((await agent.getSessionState('s1'))?.status).toBe('idle') + + const stopCalls = hookDispatcher.dispatchEvent.mock.calls.filter( + (call: any[]) => call[0] === 'Stop' + ) + const sessionEndCalls = hookDispatcher.dispatchEvent.mock.calls.filter( + (call: any[]) => call[0] === 'SessionEnd' + ) + expect(stopCalls).toHaveLength(1) + expect(stopCalls[0][1]).toMatchObject({ + conversationId: 's1', + stop: { reason: 'user_stop', userStop: true } + }) + expect(sessionEndCalls).toHaveLength(1) + expect(sessionEndCalls[0][1]).toMatchObject({ + conversationId: 's1', + usage: expectedUsage, + error: { message: 'common.error.userCanceledGeneration' } + }) + } + ) + + it('does not duplicate stream-owned assistant finalization on stop', async () => { let resolveRun: ((value: any) => void) | null = null ;(processStream as ReturnType).mockImplementationOnce( - () => + (params: any) => new Promise((resolve) => { - resolveRun = resolve + resolveRun = (value) => { + params.io.messageStore.setMessageError( + params.run.messageId, + [ + { + type: 'error', + content: 'common.error.userCanceledGeneration', + status: 'error', + timestamp: Date.now() + } + ], + JSON.stringify({ runOutcome: 'aborted', runStopReason: 'user_stop' }) + ) + resolve(value) + } }) ) sqlitePresenter.deepchatMessagesTable.get.mockReturnValue({ @@ -4892,6 +5155,7 @@ describe('AgentRuntimePresenter', () => { }) await processPromise + expect(sqlitePresenter.deepchatMessagesTable.updateContentAndStatus).toHaveBeenCalledTimes(1) const [messageId, contentJson, status] = sqlitePresenter.deepchatMessagesTable.updateContentAndStatus.mock.calls[0] expect(messageId).toBe('mock-msg-id') @@ -6500,6 +6764,34 @@ describe('AgentRuntimePresenter', () => { expect(getRuntimeState(agent, 's1').status).toBe('idle') }) + it('cancels manual compaction while tool definitions are still loading', async () => { + const toolDefinitions = deferred<[]>() + const prepareForManualCompaction = vi.spyOn( + (agent as any).compactionService, + 'prepareForManualCompaction' + ) + toolPresenter.getAllToolDefinitions.mockImplementationOnce( + async () => await toolDefinitions.promise + ) + await agent.initSession('s1', { + providerId: 'openai', + modelId: 'gpt-4', + generationSettings: { + contextLength: 128000, + maxTokens: 4096 + } + }) + + const compaction = agent.compactSession('s1') + await vi.waitFor(() => expect(toolPresenter.getAllToolDefinitions).toHaveBeenCalledTimes(1)) + + await agent.cancelGeneration('s1') + + await expect(compaction).rejects.toMatchObject({ name: 'AbortError' }) + expect(prepareForManualCompaction).not.toHaveBeenCalled() + expect(getRuntimeState(agent, 's1').status).toBe('idle') + }) + it('does not let stale manual compaction reset replacement instance resources', async () => { const preparation = deferred() vi.spyOn( @@ -6723,8 +7015,9 @@ describe('AgentRuntimePresenter', () => { expect(sqlitePresenter.deepchatMessagesTable.delete).toHaveBeenCalledWith('mock-msg-id') }) - it('treats aborted compaction signals as cancellation even for non-abort errors', async () => { + it('normalizes a late compaction failure to AbortError after cancellation', async () => { await agent.initSession('s1', { providerId: 'openai', modelId: 'gpt-4' }) + const instance = agent.deepChatRuntime.getOrHydrate(toAppSessionId('s1')) sqlitePresenter.deepchatMessagesTable.delete.mockClear() const abortController = new AbortController() @@ -6754,9 +7047,14 @@ describe('AgentRuntimePresenter', () => { }, { signal: abortController.signal } ) - ).rejects.toThrow('late failure') + ).rejects.toMatchObject({ name: 'AbortError' }) expect(sqlitePresenter.deepchatMessagesTable.delete).toHaveBeenCalledWith('mock-msg-id') + expect(instance.getCompactionState()).toEqual({ + status: 'idle', + cursorOrderSeq: 1, + summaryUpdatedAt: null + }) expectPublished('sessions.compaction.changed', { sessionId: 's1', status: 'idle', @@ -6999,6 +7297,7 @@ describe('AgentRuntimePresenter', () => { orderSeq?: number status?: 'pending' | 'sent' | 'error' blocks?: unknown[] + metadata?: Record }) => { const row = { id: overrides?.id ?? 'm1', @@ -7008,7 +7307,7 @@ describe('AgentRuntimePresenter', () => { content: JSON.stringify(overrides?.blocks ?? []), status: overrides?.status ?? 'pending', is_context_edge: 0, - metadata: '{}', + metadata: JSON.stringify(overrides?.metadata ?? {}), created_at: Date.now(), updated_at: Date.now() } @@ -7019,6 +7318,29 @@ describe('AgentRuntimePresenter', () => { return row } + const registerActiveInteractionRun = ( + messageId: string, + blocks: AssistantMessageBlock[], + runId = `run-${messageId}` + ) => { + const instance = agent.deepChatRuntime.getOrHydrate(toAppSessionId('s1')) + const abortController = new AbortController() + const streamState = createState() + streamState.blocks = structuredClone(blocks) + const run = createLoopRun({ + runId, + sessionId: toAppSessionId('s1'), + messageId, + abortController, + messages: [], + streamState, + resources: { toolDefinitions: [], activeSkillNames: [] } + }) + instance.registerActiveGeneration(run) + getRuntimeState(agent, 's1').status = 'generating' + return { instance, run, abortController, streamState } + } + it('assembles resume context after compaction and preserves base-only round refresh', async () => { const order: string[] = [] await agent.initSession('s1', { providerId: 'openai', modelId: 'gpt-4' }) @@ -7388,6 +7710,24 @@ describe('AgentRuntimePresenter', () => { expect((await agent.getSessionState('s1'))?.status).toBe('idle') }) + it('cancels resume while tool definitions are still loading', async () => { + const toolDefinitions = deferred<[]>() + toolPresenter.getAllToolDefinitions.mockImplementationOnce( + async () => await toolDefinitions.promise + ) + await agent.initSession('s1', { providerId: 'openai', modelId: 'gpt-4' }) + makeAssistantRow({ blocks: [] }) + + const resume = (agent as any).resumeAssistantMessage('s1', 'm1', []) as Promise + await vi.waitFor(() => expect(toolPresenter.getAllToolDefinitions).toHaveBeenCalledTimes(1)) + + await agent.cancelGeneration('s1') + + await expect(resume).resolves.toBe(false) + expect(processStream).not.toHaveBeenCalled() + expect((await agent.getSessionState('s1'))?.status).toBe('idle') + }) + it('does not continue a stale resume after resource loading rehydrates the session', async () => { const toolDefinitions = deferred<[]>() toolPresenter.getAllToolDefinitions.mockImplementationOnce( @@ -8211,6 +8551,68 @@ describe('AgentRuntimePresenter', () => { expect(updatedBlocks[1].extra.needsUserAction).toBe(false) }) + it('rejects deferred permission execution at the global tool-call cap', async () => { + await agent.initSession('s1', { providerId: 'openai', modelId: 'gpt-4' }) + makeAssistantRow({ + metadata: { toolCalls: 128 }, + blocks: [ + { + type: 'tool_call', + status: 'pending', + timestamp: 1, + tool_call: { id: 'tc1', name: 'write_file', params: '{"path":"a.txt"}', response: '' } + }, + { + type: 'action', + action_type: 'tool_call_permission', + status: 'pending', + timestamp: 2, + content: 'Need permission', + tool_call: { id: 'tc1', name: 'write_file', params: '{"path":"a.txt"}' }, + extra: { + needsUserAction: true, + permissionType: 'write', + permissionRequest: JSON.stringify({ + permissionType: 'write', + description: 'Need permission', + toolName: 'write_file', + serverName: 'agent-filesystem', + paths: ['a.txt'] + }) + } + } + ] + }) + + const result = await agent.respondToolInteraction('s1', 'm1', 'tc1', { + kind: 'permission', + granted: true + }) + + expect(result).toEqual({ resumed: true }) + expect(toolPresenter.callTool).not.toHaveBeenCalled() + expect(processStream).toHaveBeenCalledTimes(1) + expect( + (processStream as ReturnType).mock.calls[0][0].initialAccounting + ).toEqual(expect.objectContaining({ toolCalls: 128 })) + const updatedBlocks = JSON.parse( + sqlitePresenter.deepchatMessagesTable.updateContent.mock.calls.at(-1)[1] + ) + expect(updatedBlocks[0]).toEqual( + expect.objectContaining({ + status: 'error', + tool_call: expect.objectContaining({ + response: 'Tool call was not executed because the maximum tool-call limit was reached.' + }) + }) + ) + expect( + sqlitePresenter.deepchatMessagesTable.updateMetadata.mock.calls.every( + ([, metadata]: [string, string]) => JSON.parse(metadata).toolCalls <= 128 + ) + ).toBe(true) + }) + it('fails loudly when a confirmed permission grant has no session permission port', async () => { const skillPresenter = getSkillPresenterMock() const agentWithoutPermissionPort = new AgentRuntimePresenter( @@ -8547,7 +8949,7 @@ describe('AgentRuntimePresenter', () => { it('handles ACP permission grant through live provider resolver without deferred tool resume', async () => { await agent.initSession('s1', { providerId: 'acp', modelId: 'claude-code-acp' }) - makeAssistantRow({ + const row = makeAssistantRow({ blocks: [ { type: 'tool_call', @@ -8580,7 +8982,10 @@ describe('AgentRuntimePresenter', () => { ] }) - const instance = agent.deepChatRuntime.getOrHydrate(toAppSessionId('s1')) + const { instance } = registerActiveInteractionRun( + 'm1', + JSON.parse(row.content) as AssistantMessageBlock[] + ) instance.registerActiveProviderPermission({ requestId: 'acp-req-1', messageId: 'm1', @@ -8646,6 +9051,8 @@ describe('AgentRuntimePresenter', () => { expect(updatedBlocks[1].status).toBe('granted') expect(updatedBlocks[1].extra.needsUserAction).toBe(false) expect(instance.hasActiveProviderPermission('acp-req-1')).toBe(false) + expect(instance.getActiveGeneration()?.messageId).toBe('m1') + expect(getRuntimeState(agent, 's1').status).toBe('generating') }) it('cancels only the target session live ACP permission resolvers', async () => { @@ -8677,9 +9084,9 @@ describe('AgentRuntimePresenter', () => { expect(second.hasActiveProviderPermission('acp-req-2')).toBe(true) }) - it('falls back to direct ACP permission resolve when live resolver is missing', async () => { + it('keeps a healthy matching run active when direct ACP permission resolve succeeds', async () => { await agent.initSession('s1', { providerId: 'acp', modelId: 'claude-code-acp' }) - makeAssistantRow({ + const row = makeAssistantRow({ blocks: [ { type: 'tool_call', @@ -8711,6 +9118,10 @@ describe('AgentRuntimePresenter', () => { } ] }) + const { instance, streamState } = registerActiveInteractionRun( + 'm1', + JSON.parse(row.content) as AssistantMessageBlock[] + ) const result = await agent.respondToolInteraction('s1', 'm1', 'tc1', { kind: 'permission', @@ -8727,9 +9138,338 @@ describe('AgentRuntimePresenter', () => { expect(updatedBlocks[1].status).toBe('denied') expect(updatedBlocks[1].content).toBe('User denied the request.') expect(updatedBlocks[1].extra.needsUserAction).toBe(false) + expect(streamState.blocks[1]).toEqual( + expect.objectContaining({ + status: 'denied', + extra: expect.objectContaining({ needsUserAction: false }) + }) + ) + expect(sqlitePresenter.deepchatMessagesTable.updateStatus).toHaveBeenCalledWith( + 'm1', + 'pending' + ) + expect(sqlitePresenter.deepchatMessagesTable.updateContentAndStatus).not.toHaveBeenCalled() + expect(instance.getActiveGeneration()?.messageId).toBe('m1') + expect(getRuntimeState(agent, 's1').status).toBe('generating') + expect( + hookDispatcher.dispatchEvent.mock.calls.filter( + ([event]) => event === 'Stop' || event === 'SessionEnd' + ) + ).toHaveLength(0) }) - it('clears stale ACP permission modals when the provider request no longer exists', async () => { + it('leaves settlement to an already-aborted matching ACP run', async () => { + await agent.initSession('s1', { providerId: 'acp', modelId: 'claude-code-acp' }) + const row = makeAssistantRow({ + blocks: [ + { + type: 'tool_call', + status: 'pending', + timestamp: 1, + tool_call: { id: 'tc1', name: 'Terminal', params: '{}', response: '' } + }, + { + type: 'action', + action_type: 'tool_call_permission', + status: 'pending', + timestamp: 2, + content: 'Need permission', + tool_call: { id: 'tc1', name: 'Terminal', params: '{}' }, + extra: { + needsUserAction: true, + permissionType: 'command', + providerId: 'acp', + permissionRequestId: 'acp-aborted-req', + permissionRequest: JSON.stringify({ + permissionType: 'command', + description: 'Need permission', + providerId: 'acp', + requestId: 'acp-aborted-req' + }) + } + } + ] + }) + const { instance, abortController } = registerActiveInteractionRun( + 'm1', + JSON.parse(row.content) as AssistantMessageBlock[] + ) + abortController.abort() + + await expect( + agent.respondToolInteraction('s1', 'm1', 'tc1', { + kind: 'permission', + granted: true + }) + ).resolves.toEqual({ resumed: false }) + + expect(llmProvider.resolveAgentPermission).not.toHaveBeenCalled() + expect(sqlitePresenter.deepchatMessagesTable.updateContentAndStatus).not.toHaveBeenCalled() + expect(instance.getActiveGeneration()?.runId).toBe('run-m1') + expect( + hookDispatcher.dispatchEvent.mock.calls.filter( + ([event]) => event === 'Stop' || event === 'SessionEnd' + ) + ).toHaveLength(0) + }) + + it('fails an orphaned ACP permission closed without overwriting a newer run', async () => { + await agent.initSession('s1', { providerId: 'acp', modelId: 'claude-code-acp' }) + makeAssistantRow({ + metadata: { + runId: 'old-run', + runOutcome: 'paused', + runStopReason: 'interaction', + inputTokens: 7, + outputTokens: 5, + totalTokens: 12 + }, + blocks: [ + { + type: 'tool_call', + status: 'pending', + timestamp: 1, + tool_call: { id: 'tc1', name: 'Terminal', params: '{}', response: '' } + }, + { + type: 'action', + action_type: 'tool_call_permission', + status: 'pending', + timestamp: 2, + content: 'Need permission', + tool_call: { id: 'tc1', name: 'Terminal', params: '{}' }, + extra: { + needsUserAction: true, + permissionType: 'command', + providerId: 'acp', + permissionRequestId: 'acp-orphan-req', + permissionRequest: JSON.stringify({ + permissionType: 'command', + description: 'Need permission', + providerId: 'acp', + requestId: 'acp-orphan-req' + }) + } + } + ] + }) + const { instance, abortController } = registerActiveInteractionRun( + 'new-message', + [], + 'new-run' + ) + + const result = await agent.respondToolInteraction('s1', 'm1', 'tc1', { + kind: 'permission', + granted: true + }) + + expect(result).toEqual({ resumed: false }) + expect(llmProvider.resolveAgentPermission).toHaveBeenCalledWith('acp-orphan-req', false) + const errorWrite = + sqlitePresenter.deepchatMessagesTable.updateContentAndStatus.mock.calls.find( + (call) => call[0] === 'm1' && call[2] === 'error' + ) + expect(errorWrite).toBeDefined() + const terminalBlocks = JSON.parse(errorWrite?.[1] ?? '[]') + expect(terminalBlocks[0]).toEqual( + expect.objectContaining({ + status: 'error', + tool_call: expect.objectContaining({ + response: 'ACP permission request lost its active generation.' + }) + }) + ) + expect(terminalBlocks[1]).toEqual( + expect.objectContaining({ + status: 'error', + content: 'ACP permission request lost its active generation.', + extra: expect.objectContaining({ needsUserAction: false }) + }) + ) + expect(JSON.parse(errorWrite?.[3] ?? '{}')).toEqual( + expect.objectContaining({ + runId: 'old-run', + runOutcome: 'error', + runStopReason: 'provider_error', + totalTokens: 12 + }) + ) + expect(hookDispatcher.dispatchEvent).toHaveBeenCalledWith( + 'Stop', + expect.objectContaining({ + stop: expect.objectContaining({ reason: 'provider_error', userStop: false }) + }) + ) + expect(hookDispatcher.dispatchEvent).toHaveBeenCalledWith( + 'SessionEnd', + expect.objectContaining({ + usage: expect.objectContaining({ inputTokens: 7, outputTokens: 5, totalTokens: 12 }), + error: expect.objectContaining({ + message: 'ACP permission request lost its active generation.' + }) + }) + ) + expect(instance.getActiveGeneration()?.runId).toBe('new-run') + expect(instance.getAbortController()).toBe(abortController) + expect(abortController.signal.aborted).toBe(false) + expect(getRuntimeState(agent, 's1').status).toBe('generating') + }) + + it('does not overwrite an aborted orphan when ACP permission resolution arrives late', async () => { + await agent.initSession('s1', { providerId: 'acp', modelId: 'claude-code-acp' }) + const permissionResolution = deferred() + llmProvider.resolveAgentPermission.mockReturnValueOnce(permissionResolution.promise) + makeAssistantRow({ + metadata: { + runId: 'orphan-run', + inputTokens: 4, + outputTokens: 3, + totalTokens: 7, + runOutcome: 'paused', + runStopReason: 'interaction' + }, + blocks: [ + { + type: 'tool_call', + status: 'pending', + timestamp: 1, + tool_call: { id: 'tc-late', name: 'Terminal', params: '{}', response: '' } + }, + { + type: 'action', + action_type: 'tool_call_permission', + status: 'pending', + timestamp: 2, + content: 'Need permission', + tool_call: { id: 'tc-late', name: 'Terminal', params: '{}' }, + extra: { + needsUserAction: true, + permissionType: 'command', + providerId: 'acp', + permissionRequestId: 'acp-late-req', + permissionRequest: JSON.stringify({ + permissionType: 'command', + description: 'Need permission', + providerId: 'acp', + requestId: 'acp-late-req' + }) + } + } + ] + }) + + const response = agent.respondToolInteraction('s1', 'm1', 'tc-late', { + kind: 'permission', + granted: true + }) + await vi.waitFor(() => + expect(llmProvider.resolveAgentPermission).toHaveBeenCalledWith('acp-late-req', false) + ) + + await agent.cancelGeneration('s1') + await expect(response).resolves.toEqual({ resumed: false }) + + permissionResolution.resolve() + await new Promise((resolve) => setImmediate(resolve)) + + const terminalWrites = + sqlitePresenter.deepchatMessagesTable.updateContentAndStatus.mock.calls.filter( + (call) => call[0] === 'm1' && call[2] === 'error' + ) + expect(terminalWrites).toHaveLength(1) + expect(JSON.parse(terminalWrites[0][3])).toEqual( + expect.objectContaining({ + runId: 'orphan-run', + runOutcome: 'aborted', + runStopReason: 'user_stop', + totalTokens: 7 + }) + ) + expect( + hookDispatcher.dispatchEvent.mock.calls.filter(([event]) => event === 'Stop') + ).toHaveLength(1) + expect( + hookDispatcher.dispatchEvent.mock.calls.filter(([event]) => event === 'SessionEnd') + ).toHaveLength(1) + expect(getRuntimeState(agent, 's1').status).toBe('idle') + }) + + it('does not resolve a same-id ACP permission owned by another interaction', async () => { + await agent.initSession('s1', { providerId: 'acp', modelId: 'claude-code-acp' }) + makeAssistantRow({ + blocks: [ + { + type: 'tool_call', + status: 'pending', + timestamp: 1, + tool_call: { id: 'tc-old', name: 'Terminal', params: '{}', response: '' } + }, + { + type: 'action', + action_type: 'tool_call_permission', + status: 'pending', + timestamp: 2, + content: 'Old permission', + tool_call: { id: 'tc-old', name: 'Terminal', params: '{}' }, + extra: { + needsUserAction: true, + permissionType: 'command', + providerId: 'acp', + permissionRequestId: 'acp-shared-req', + permissionRequest: JSON.stringify({ + permissionType: 'command', + description: 'Old permission', + providerId: 'acp', + requestId: 'acp-shared-req' + }) + } + } + ] + }) + const { instance, abortController } = registerActiveInteractionRun( + 'new-message', + [], + 'new-run' + ) + const resolveNewPermission = vi.fn().mockResolvedValue(undefined) + instance.registerActiveProviderPermission({ + requestId: 'acp-shared-req', + messageId: 'new-message', + toolCallId: 'tc-new', + providerId: 'acp', + permissionType: 'command', + resolve: resolveNewPermission + }) + + await expect( + agent.respondToolInteraction('s1', 'm1', 'tc-old', { + kind: 'permission', + granted: true + }) + ).resolves.toEqual({ resumed: false }) + + expect(resolveNewPermission).not.toHaveBeenCalled() + expect(llmProvider.resolveAgentPermission).not.toHaveBeenCalled() + expect(instance.hasActiveProviderPermission('acp-shared-req')).toBe(true) + expect(instance.getActiveGeneration()?.runId).toBe('new-run') + expect(instance.getAbortController()).toBe(abortController) + expect(abortController.signal.aborted).toBe(false) + expect(getRuntimeState(agent, 's1').status).toBe('generating') + const errorWrite = + sqlitePresenter.deepchatMessagesTable.updateContentAndStatus.mock.calls.find( + (call) => call[0] === 'm1' && call[2] === 'error' + ) + expect(errorWrite).toBeDefined() + expect(JSON.parse(errorWrite?.[3] ?? '{}')).toEqual( + expect.objectContaining({ + runOutcome: 'error', + runStopReason: 'provider_error' + }) + ) + }) + + it('terminalizes a stale unowned ACP permission as provider_error', async () => { await agent.initSession('s1', { providerId: 'acp', modelId: 'claude-code-acp' }) getRuntimeState(agent, 's1').status = 'generating' llmProvider.resolveAgentPermission.mockRejectedValueOnce( @@ -8774,17 +9514,27 @@ describe('AgentRuntimePresenter', () => { }) expect(result).toEqual({ resumed: false }) - expect(llmProvider.resolveAgentPermission).toHaveBeenCalledWith('acp-stale-req', true) + expect(llmProvider.resolveAgentPermission).toHaveBeenCalledWith('acp-stale-req', false) expect(toolPresenter.callTool).not.toHaveBeenCalled() expect(processStream).not.toHaveBeenCalled() - const updatedBlocks = JSON.parse( - sqlitePresenter.deepchatMessagesTable.updateContent.mock.calls[0][1] - ) - expect(updatedBlocks[1].status).toBe('denied') + const errorWrite = + sqlitePresenter.deepchatMessagesTable.updateContentAndStatus.mock.calls.find( + (call) => call[0] === 'm1' && call[2] === 'error' + ) + expect(errorWrite).toBeDefined() + const updatedBlocks = JSON.parse(errorWrite?.[1] ?? '[]') + expect(updatedBlocks[0].status).toBe('error') + expect(updatedBlocks[0].tool_call.response).toBe('Permission request expired.') + expect(updatedBlocks[1].status).toBe('error') expect(updatedBlocks[1].content).toBe('Permission request expired.') expect(updatedBlocks[1].extra.needsUserAction).toBe(false) - expect(sqlitePresenter.deepchatMessagesTable.updateStatus).toHaveBeenCalledWith('m1', 'sent') - expect(getRuntimeState(agent, 's1').status).toBe('idle') + expect(JSON.parse(errorWrite?.[3] ?? '{}')).toEqual( + expect.objectContaining({ + runOutcome: 'error', + runStopReason: 'provider_error' + }) + ) + expect(getRuntimeState(agent, 's1').status).toBe('error') }) }) @@ -8813,6 +9563,20 @@ describe('AgentRuntimePresenter', () => { ) }) + it('does not publish permission mode when persistence fails', async () => { + await agent.initSession('s1', { + providerId: 'openai', + modelId: 'gpt-4', + permissionMode: 'full_access' + }) + sqlitePresenter.deepchatSessionsTable.updatePermissionMode.mockImplementationOnce(() => { + throw new Error('write failed') + }) + + await expect(agent.setPermissionMode('s1', 'default')).rejects.toThrow('write failed') + expect(getRuntimeState(agent, 's1').permissionMode).toBe('full_access') + }) + it('getPermissionMode falls back to db session row', async () => { sqlitePresenter.deepchatSessionsTable.get.mockReturnValue({ id: 's2', @@ -8825,6 +9589,17 @@ describe('AgentRuntimePresenter', () => { expect(mode).toBe('default') }) + it('normalizes an unknown persisted permission mode to default', async () => { + sqlitePresenter.deepchatSessionsTable.get.mockReturnValue({ + id: 's2', + provider_id: 'openai', + model_id: 'gpt-4', + permission_mode: 'unknown' + }) + + await expect(agent.getPermissionMode('s2')).resolves.toBe('default') + }) + it('falls back to ask_user when auto-review returns invalid JSON', async () => { llmProvider.generateCompletionStandalone.mockResolvedValueOnce('not json') @@ -8891,6 +9666,50 @@ describe('AgentRuntimePresenter', () => { ) }) + it.each([ + ['missing', undefined], + ['invalid', 'unknown'] + ])('falls back to ask_user when auto-review risk level is %s', async (_label, riskLevel) => { + llmProvider.generateCompletionStandalone.mockImplementationOnce( + async (_provider, messages) => { + const prompt = String(messages[1]?.content ?? '') + const actionHash = prompt.match(/"actionHash": "([^"]+)"/)?.[1] ?? '' + return JSON.stringify({ + actionHash, + decision: 'auto_allow', + riskLevel, + userAuthorization: 'high', + rationale: 'safe' + }) + } + ) + + const result = await (agent as any).reviewToolPermissionForAutoApprove( + { + sessionId: 's1', + messageId: 'm1', + toolCallId: 'tc1', + toolName: 'read', + toolArgs: '{"path":"/tmp/a.txt"}', + toolSource: 'agent', + reason: 'tool_call' + }, + { + providerId: 'openai', + modelId: 'gpt-4', + messages: [{ role: 'user', content: 'read /tmp/a.txt' }], + signal: new AbortController().signal + } + ) + + expect(result).toEqual( + expect.objectContaining({ + decision: 'ask_user', + rationale: 'Auto-review returned an invalid risk level.' + }) + ) + }) + it('falls back to ask_user when auto-review times out', async () => { vi.useFakeTimers() llmProvider.generateCompletionStandalone.mockImplementationOnce( @@ -9037,6 +9856,37 @@ describe('AgentRuntimePresenter', () => { ) }) + it('returns a deferred tool-local AbortError as a tool failure while the run is active', async () => { + toolPresenter.getAllToolDefinitions.mockResolvedValueOnce([ + { + type: 'function', + function: { + name: 'echo', + description: 'Echo tool', + parameters: { type: 'object', properties: {} } + }, + server: { name: 'test-server', icons: '', description: '' } + } + ]) + const timeoutError = new Error('Model request timed out') + timeoutError.name = 'AbortError' + toolPresenter.callTool.mockRejectedValueOnce(timeoutError) + + await agent.initSession('s1', { providerId: 'openai', modelId: 'gpt-4' }) + + const result = await (agent as any).executeDeferredToolCall('s1', 'm1', { + id: 'tc1', + name: 'echo', + params: '{}' + }) + + expect(result).toEqual({ + responseText: 'Error: Model request timed out', + isError: true, + invoked: true + }) + }) + it('returns image previews from deferred tool execution', async () => { toolPresenter.getAllToolDefinitions.mockResolvedValueOnce([ { @@ -9329,12 +10179,7 @@ describe('AgentRuntimePresenter', () => { await agent.cancelGeneration('s1') expect(capturedSignal?.aborted).toBe(true) - await expect(executionPromise).resolves.toEqual( - expect.objectContaining({ - isError: true, - responseText: 'Error: Aborted' - }) - ) + await expect(executionPromise).rejects.toMatchObject({ name: 'AbortError' }) expect( agent.deepChatRuntime .getHydrated(toAppSessionId('s1')) diff --git a/test/main/presenter/agentRuntimePresenter/compactionService.test.ts b/test/main/presenter/agentRuntimePresenter/compactionService.test.ts index 374b4672c..cad9efa2e 100644 --- a/test/main/presenter/agentRuntimePresenter/compactionService.test.ts +++ b/test/main/presenter/agentRuntimePresenter/compactionService.test.ts @@ -771,6 +771,44 @@ describe('CompactionService', () => { expect(llmProviderPresenter.generateText).not.toHaveBeenCalled() }) + it('does not persist a summary when cancellation arrives during the summary LLM call', async () => { + const { service, sessionStore, llmProviderPresenter } = createService() + const abortController = new AbortController() + let resolveSummary!: (value: { content: string }) => void + llmProviderPresenter.generateText.mockReturnValue( + new Promise<{ content: string }>((resolve) => { + resolveSummary = resolve + }) + ) + + const compactionPromise = service.applyCompaction( + { + sessionId: 's1', + previousState: { + summaryText: null, + summaryCursorOrderSeq: 1, + summaryUpdatedAt: null + }, + targetCursorOrderSeq: 3, + summaryBlocks: ['span to summarize'], + currentModel: { + providerId: 'openai', + modelId: 'gpt-4o', + contextLength: 4096 + }, + reserveTokens: 512 + }, + abortController.signal + ) + await vi.waitFor(() => expect(llmProviderPresenter.generateText).toHaveBeenCalled()) + + abortController.abort() + resolveSummary({ content: 'late generated summary' }) + + await expect(compactionPromise).rejects.toMatchObject({ name: 'AbortError' }) + expect(sessionStore.compareAndSetSummaryState).not.toHaveBeenCalled() + }) + it('avoids direct oversized single-shot summarization when splitLargeBlock does not split', async () => { const { service } = createService() const generateSummaryTextSpy = vi diff --git a/test/main/presenter/agentRuntimePresenter/dispatch.test.ts b/test/main/presenter/agentRuntimePresenter/dispatch.test.ts index 8e89482a8..3cc49a32c 100644 --- a/test/main/presenter/agentRuntimePresenter/dispatch.test.ts +++ b/test/main/presenter/agentRuntimePresenter/dispatch.test.ts @@ -1766,7 +1766,7 @@ describe('dispatch', () => { expect(toolPresenter.preCheckToolPermission).toHaveBeenCalledWith( expect.objectContaining({ id: 'tc-write' }), - { permissionMode: 'full_access' } + { permissionMode: 'full_access', signal: io.abortSignal } ) expect(hooks.reviewToolPermission).toHaveBeenCalledWith( expect.objectContaining({ @@ -2320,10 +2320,10 @@ describe('dispatch', () => { expect(block!.status).toBe('error') }) - it('stops on abort', async () => { + it('commits a returned parallel result before stopping on abort', async () => { const abortController = new AbortController() const abortIo = createIo({ abortSignal: abortController.signal }) - const tools = [makeTool('tool_a'), makeTool('tool_b')] + const tools = [makeAgentTool('read')] const toolPresenter = createMockToolPresenter() // Abort after first tool call @@ -2337,23 +2337,24 @@ describe('dispatch', () => { content: '', status: 'pending', timestamp: Date.now(), - tool_call: { id: 'tc1', name: 'tool_a', params: '{}', response: '' } + tool_call: { id: 'tc1', name: 'read', params: '{"path":"a"}', response: '' } }) state.blocks.push({ type: 'tool_call', content: '', status: 'pending', timestamp: Date.now(), - tool_call: { id: 'tc2', name: 'tool_b', params: '{}', response: '' } + tool_call: { id: 'tc2', name: 'read', params: '{"path":"b"}', response: '' } }) state.completedToolCalls = [ - { id: 'tc1', name: 'tool_a', arguments: '{}' }, - { id: 'tc2', name: 'tool_b', arguments: '{}' } + { id: 'tc1', name: 'read', arguments: '{"path":"a"}' }, + { id: 'tc2', name: 'read', arguments: '{"path":"b"}' } ] - const executed = await executeTools( + const conversation: any[] = [] + const executing = executeTools( state, - [], + conversation, 0, tools, toolPresenter, @@ -2365,9 +2366,138 @@ describe('dispatch', () => { 1024 ) - // Only first tool should have been called - expect(executed.executed).toBe(1) + await expect(executing).resolves.toMatchObject({ type: 'completed', executed: 1 }) expect(toolPresenter.callTool).toHaveBeenCalledTimes(1) + expect(conversation).toEqual( + expect.arrayContaining([ + expect.objectContaining({ role: 'tool', tool_call_id: 'tc1', content: 'ok' }) + ]) + ) + expect(state.blocks.find((block) => block.tool_call?.id === 'tc1')).toMatchObject({ + status: 'success', + tool_call: { response: 'ok' } + }) + expect(state.blocks.find((block) => block.tool_call?.id === 'tc2')).toMatchObject({ + status: 'pending', + tool_call: { response: '' } + }) + }) + + it('stages CanceledError from a parallel read batch when the run remains active', async () => { + const tools = [makeAgentTool('read')] + const toolPresenter = createMockToolPresenter() + const canceledError = new Error('Canceled') + canceledError.name = 'CanceledError' + ;(toolPresenter.callTool as ReturnType) + .mockRejectedValueOnce(canceledError) + .mockResolvedValueOnce({ + content: 'second result', + rawData: { toolCallId: 'tc2', content: 'second result', isError: false } + }) + + state.blocks.push( + { + type: 'tool_call', + content: '', + status: 'pending', + timestamp: Date.now(), + tool_call: { id: 'tc1', name: 'read', params: '{"path":"a"}', response: '' } + }, + { + type: 'tool_call', + content: '', + status: 'pending', + timestamp: Date.now(), + tool_call: { id: 'tc2', name: 'read', params: '{"path":"b"}', response: '' } + } + ) + state.completedToolCalls = [ + { id: 'tc1', name: 'read', arguments: '{"path":"a"}' }, + { id: 'tc2', name: 'read', arguments: '{"path":"b"}' } + ] + const conversation: any[] = [] + + await expect( + executeTools( + state, + conversation, + 0, + tools, + toolPresenter, + 'gpt-4', + io, + 'full_access', + new ToolOutputGuard(), + 32000, + 1024 + ) + ).resolves.toMatchObject({ type: 'completed', executed: 2 }) + expect(conversation).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + role: 'tool', + tool_call_id: 'tc1', + content: 'Error: Canceled' + }), + expect.objectContaining({ + role: 'tool', + tool_call_id: 'tc2', + content: 'second result' + }) + ]) + ) + expect(state.blocks.find((block) => block.tool_call?.id === 'tc1')).toMatchObject({ + status: 'error', + tool_call: { response: 'Error: Canceled' } + }) + }) + + it('commits the raw result when cancellation wins during asynchronous normalization', async () => { + const abortController = new AbortController() + const abortIo = createIo({ abortSignal: abortController.signal }) + const tools = [makeTool('tool_a')] + const toolPresenter = createMockToolPresenter({ tool_a: 'raw result' }) + const conversation: any[] = [] + const resultNormalizer = vi.fn(async () => { + abortController.abort() + return 'normalized result' + }) + + state.blocks.push({ + type: 'tool_call', + content: '', + status: 'pending', + timestamp: Date.now(), + tool_call: { id: 'tc1', name: 'tool_a', params: '{}', response: '' } + }) + state.completedToolCalls = [{ id: 'tc1', name: 'tool_a', arguments: '{}' }] + + await expect( + executeTools( + state, + conversation, + 0, + tools, + toolPresenter, + 'gpt-4', + abortIo, + 'full_access', + new ToolOutputGuard(), + 32000, + 1024, + { resultNormalizer } + ) + ).resolves.toMatchObject({ type: 'completed', executed: 1 }) + expect(resultNormalizer).toHaveBeenCalledOnce() + expect(conversation).toEqual( + expect.arrayContaining([ + expect.objectContaining({ role: 'tool', tool_call_id: 'tc1', content: 'raw result' }) + ]) + ) + expect(state.blocks[0]).toMatchObject({ + status: 'success', + tool_call: { response: 'raw result' } + }) }) it('flushes to renderer and DB after each tool execution', async () => { diff --git a/test/main/presenter/agentRuntimePresenter/messageStore.test.ts b/test/main/presenter/agentRuntimePresenter/messageStore.test.ts index e55aca96a..6fac17307 100644 --- a/test/main/presenter/agentRuntimePresenter/messageStore.test.ts +++ b/test/main/presenter/agentRuntimePresenter/messageStore.test.ts @@ -19,6 +19,7 @@ function createMockSqlitePresenter() { insert: vi.fn(), updateContent: vi.fn(), updateStatus: vi.fn(), + updateMetadata: vi.fn(), updateContentAndStatus: vi.fn(), getBySession: vi.fn().mockReturnValue([]), hasBySession: vi.fn().mockReturnValue(false), @@ -202,6 +203,64 @@ describe('DeepChatMessageStore', () => { ) expect(sqlitePresenter.deepchatMessagesTable.updateContent).not.toHaveBeenCalled() }) + + it('persists run metadata while keeping an interaction-paused message pending', () => { + sqlitePresenter.deepchatMessagesTable.get.mockReturnValue({ + id: 'm1', + session_id: 's1', + role: 'assistant', + created_at: 1000, + updated_at: 2000 + }) + sqlitePresenter.deepchatSessionsTable.get.mockReturnValue({ + provider_id: 'openai', + model_id: 'gpt-4o' + }) + const blocks = [ + { + type: 'content' as const, + content: 'partial', + status: 'success' as const, + timestamp: 1000 + } + ] + const metadata = '{"runOutcome":"paused","inputTokens":10,"outputTokens":2,"totalTokens":12}' + + store.updateAssistantContent('m1', blocks, metadata) + + expect(sqlitePresenter.deepchatMessagesTable.updateMetadata).toHaveBeenCalledWith( + 'm1', + metadata + ) + expect(sqlitePresenter.deepchatMessagesTable.updateStatus).toHaveBeenCalledWith( + 'm1', + 'pending' + ) + expect(sqlitePresenter.deepchatUsageStatsTable.upsert).toHaveBeenCalledWith( + expect.objectContaining({ + messageId: 'm1', + inputTokens: 10, + outputTokens: 2, + totalTokens: 12, + source: 'live' + }) + ) + }) + }) + + describe('updateAssistantMetadata', () => { + it('persists accounting without rewriting assistant blocks or status', () => { + const metadata = '{"runOutcome":"paused","toolCalls":4}' + + store.updateAssistantMetadata('m1', metadata) + + expect(sqlitePresenter.deepchatMessagesTable.updateMetadata).toHaveBeenCalledWith( + 'm1', + metadata + ) + expect(sqlitePresenter.deepchatAssistantBlocksTable.replaceForMessage).not.toHaveBeenCalled() + expect(sqlitePresenter.deepchatMessagesTable.updateStatus).not.toHaveBeenCalled() + }) }) describe('finalizeAssistantMessage', () => { diff --git a/test/main/presenter/agentRuntimePresenter/noProgressToolLoopGuard.test.ts b/test/main/presenter/agentRuntimePresenter/noProgressToolLoopGuard.test.ts new file mode 100644 index 000000000..e9264def6 --- /dev/null +++ b/test/main/presenter/agentRuntimePresenter/noProgressToolLoopGuard.test.ts @@ -0,0 +1,221 @@ +import { describe, expect, it } from 'vitest' +import type { ChatMessage } from '@shared/types/core/chat-message' +import { + extractLatestCompletedToolBatch, + NoProgressToolLoopGuard +} from '@/presenter/agentRuntimePresenter/noProgressToolLoopGuard' + +function toolCall(id: string) { + return { id, name: 'read', arguments: '{"path":"README.md"}' } +} + +function toolMessage(id: string, content: string): ChatMessage { + return { role: 'tool', tool_call_id: id, content } +} + +describe('NoProgressToolLoopGuard', () => { + it('ignores volatile timestamps and generated IDs in otherwise identical results', () => { + const guard = new NoProgressToolLoopGuard() + const first = guard.observe( + [toolCall('call-1')], + [ + toolMessage( + 'call-1', + JSON.stringify({ + path: 'README.md', + updatedAt: '2026-07-13T10:00:00.000Z', + requestId: '3e9b1f2d-b83d-4c7c-9c39-5d3a35f0a920', + toolCallId: 'call-1', + content: 'unchanged' + }) + ) + ] + ) + const secondMessages = [ + toolMessage( + 'call-2', + JSON.stringify({ + requestId: 'cbf1a435-68a5-4f30-bbf8-42ec2c0ce6fe', + toolCallId: 'call-2', + content: 'unchanged', + updatedAt: '2026-07-13T10:01:00.000Z', + path: 'README.md' + }) + ) + ] + const second = guard.observe([toolCall('call-2')], secondMessages) + + expect(first.repeatedBatchCount).toBe(1) + expect(second.repeatedBatchCount).toBe(2) + expect(second.correctionAppended).toBe(true) + expect(secondMessages[0].content).toContain('agent_no_progress') + }) + + it('preserves UUIDs in semantic result fields as progress', () => { + const guard = new NoProgressToolLoopGuard() + const first = guard.observe( + [toolCall('call-1')], + [ + toolMessage( + 'call-1', + JSON.stringify({ + recordId: '3e9b1f2d-b83d-4c7c-9c39-5d3a35f0a920', + status: 'created' + }) + ) + ] + ) + const second = guard.observe( + [toolCall('call-2')], + [ + toolMessage( + 'call-2', + JSON.stringify({ + recordId: 'cbf1a435-68a5-4f30-bbf8-42ec2c0ce6fe', + status: 'created' + }) + ) + ] + ) + + expect(first.snapshot.fingerprint).not.toBe(second.snapshot.fingerprint) + expect(second).toMatchObject({ repeatedBatchCount: 1, correctionAppended: false }) + }) + + it('normalizes timestamps and explicitly labeled generated IDs in unstructured text', () => { + const guard = new NoProgressToolLoopGuard() + guard.observe( + [toolCall('call-1')], + [ + toolMessage( + 'call-1', + 'unchanged\nrequest 3e9b1f2d-b83d-4c7c-9c39-5d3a35f0a920\nfinished 2026-07-13T10:00:00.000Z' + ) + ] + ) + const secondMessages = [ + toolMessage( + 'call-2', + 'unchanged\nrequest cbf1a435-68a5-4f30-bbf8-42ec2c0ce6fe\nfinished 2026-07-13T10:01:00.000Z' + ) + ] + const second = guard.observe([toolCall('call-2')], secondMessages) + + expect(second).toMatchObject({ repeatedBatchCount: 2, correctionAppended: true }) + }) + + it('normalizes volatile strings nested in structured results', () => { + const guard = new NoProgressToolLoopGuard() + guard.observe( + [toolCall('call-1')], + [ + toolMessage( + 'call-1', + JSON.stringify({ + content: [ + 'unchanged', + 'finished 2026-07-13T10:00:00.000Z', + 'request 3e9b1f2d-b83d-4c7c-9c39-5d3a35f0a920' + ] + }) + ) + ] + ) + const secondMessages = [ + toolMessage( + 'call-2', + JSON.stringify({ + content: [ + 'unchanged', + 'finished 2026-07-13T10:01:00.000Z', + 'request cbf1a435-68a5-4f30-bbf8-42ec2c0ce6fe' + ] + }) + ) + ] + + const second = guard.observe([toolCall('call-2')], secondMessages) + + expect(second).toMatchObject({ repeatedBatchCount: 2, correctionAppended: true }) + }) + + it('treats changing business UUIDs in unstructured results as progress', () => { + const guard = new NoProgressToolLoopGuard() + const first = guard.observe( + [toolCall('call-1')], + [toolMessage('call-1', 'Created record 3e9b1f2d-b83d-4c7c-9c39-5d3a35f0a920')] + ) + const second = guard.observe( + [toolCall('call-2')], + [toolMessage('call-2', 'Created record cbf1a435-68a5-4f30-bbf8-42ec2c0ce6fe')] + ) + + expect(first.snapshot.fingerprint).not.toBe(second.snapshot.fingerprint) + expect(second).toMatchObject({ repeatedBatchCount: 1, correctionAppended: false }) + }) + + it('replaces the corrected tool message instead of mutating a shared message object', () => { + const guard = new NoProgressToolLoopGuard() + guard.observe([toolCall('call-1')], [toolMessage('call-1', 'stable file contents')]) + const sourceMessages = [toolMessage('call-2', 'stable file contents')] + const shallowCopy = sourceMessages.slice() + + const observation = guard.observe([toolCall('call-2')], shallowCopy) + + expect(observation.correctionAppended).toBe(true) + expect(sourceMessages[0].content).toBe('stable file contents') + expect(shallowCopy[0]).not.toBe(sourceMessages[0]) + expect(shallowCopy[0].content).toContain('agent_no_progress') + }) + + it('corrects but does not hard-stop repeated acknowledgement-only results', () => { + const guard = new NoProgressToolLoopGuard() + const observations = Array.from({ length: 6 }, (_, index) => + guard.observe([toolCall(`call-${index}`)], [toolMessage(`call-${index}`, 'ok')]) + ) + + expect(observations[1]).toMatchObject({ + repeatedBatchCount: 2, + correctionAppended: true, + shouldTerminate: false, + snapshot: { evidence: 'weak' } + }) + expect(observations.at(-1)).toMatchObject({ + repeatedBatchCount: 6, + shouldTerminate: false, + snapshot: { evidence: 'weak' } + }) + }) + + it('restores a persisted streak and observes the completed batch after a pause', () => { + const initialGuard = new NoProgressToolLoopGuard() + const initial = initialGuard.observe( + [toolCall('call-1')], + [toolMessage('call-1', 'stable file contents')] + ) + const resumeMessages: ChatMessage[] = [ + { role: 'user', content: 'Inspect README.md' }, + { + role: 'assistant', + content: '', + tool_calls: [ + { + id: 'call-2', + type: 'function', + function: { name: 'read', arguments: '{"path":"README.md"}' } + } + ] + }, + toolMessage('call-2', 'stable file contents') + ] + const resumedBatch = extractLatestCompletedToolBatch(resumeMessages) + const resumedGuard = new NoProgressToolLoopGuard(initial.snapshot) + const resumed = resumedGuard.observe( + resumedBatch?.toolCalls ?? [], + resumedBatch?.batchMessages ?? [] + ) + + expect(resumedBatch?.toolCalls).toHaveLength(1) + expect(resumed).toMatchObject({ repeatedBatchCount: 2, correctionAppended: true }) + }) +}) diff --git a/test/main/presenter/agentRuntimePresenter/process.test.ts b/test/main/presenter/agentRuntimePresenter/process.test.ts index 7b6f76f6b..ee23f7aee 100644 --- a/test/main/presenter/agentRuntimePresenter/process.test.ts +++ b/test/main/presenter/agentRuntimePresenter/process.test.ts @@ -101,6 +101,26 @@ function createMockToolPresenter(responses: Record = {}): IToolP } as unknown as IToolPresenter } +function createPostCallPermissionToolPresenter(): IToolPresenter { + const toolPresenter = createMockToolPresenter() + ;(toolPresenter.callTool as ReturnType).mockResolvedValue({ + content: 'permission required', + rawData: { + content: 'permission required', + isError: true, + requiresPermission: true, + permissionRequest: { + permissionType: 'write', + description: 'Need permission to write the file', + toolName: 'action', + serverName: 'test-server', + paths: ['output.txt'] + } + } + }) + return toolPresenter +} + function makeStreamEvents(...events: LLMCoreStreamEvent[]): LLMCoreStreamEvent[] { return events } @@ -510,6 +530,135 @@ describe('processStream', () => { expect(tapeRecorder.appendToolFact).not.toHaveBeenCalled() }) + it('accounts for executed tools before pausing a mixed tool batch', async () => { + const coreStream = vi.fn(async function* () { + yield { + type: 'tool_call_start', + tool_call_id: 'action-1', + tool_call_name: 'action' + } as LLMCoreStreamEvent + yield { + type: 'tool_call_end', + tool_call_id: 'action-1', + tool_call_arguments_complete: '{}' + } as LLMCoreStreamEvent + yield { + type: 'tool_call_start', + tool_call_id: 'question-1', + tool_call_name: 'deepchat_question' + } as LLMCoreStreamEvent + yield { + type: 'tool_call_end', + tool_call_id: 'question-1', + tool_call_arguments_complete: JSON.stringify({ + question: 'Continue?', + options: [{ label: 'Yes' }, { label: 'No' }] + }) + } as LLMCoreStreamEvent + yield { type: 'stop', stop_reason: 'tool_use' } as LLMCoreStreamEvent + }) as unknown as ProcessParams['coreStream'] + const toolPresenter = createMockToolPresenter({ action: 'ok' }) + + const result = await processStream( + createParams({ + coreStream, + toolExecution: createToolExecutionPort(toolPresenter), + tools: [makeTool('action'), makeTool('deepchat_question')] + }) + ) + + expect(result).toMatchObject({ + status: 'paused', + pendingInteractions: [expect.objectContaining({ toolCallId: 'question-1' })] + }) + expect(toolPresenter.callTool).toHaveBeenCalledOnce() + const finalPauseCall = messageStore.updateAssistantContent.mock.calls.findLast( + (call) => typeof call[2] === 'string' + ) + expect(finalPauseCall).toBeDefined() + expect(JSON.parse(finalPauseCall?.[2])).toMatchObject({ + providerRounds: 1, + toolCalls: 1, + runOutcome: 'paused', + runStopReason: 'interaction' + }) + }) + + it('counts a post-call permission tool before persisting pause', async () => { + const toolPresenter = createPostCallPermissionToolPresenter() + + const result = await processStream( + createParams({ + coreStream: createToolRoundStream('action'), + toolExecution: createToolExecutionPort(toolPresenter), + tools: [makeTool('action')], + permissionMode: 'default' + }) + ) + + expect(result).toMatchObject({ + status: 'paused', + pendingInteractions: [expect.objectContaining({ origin: 'post-call-permission' })] + }) + expect(toolPresenter.callTool).toHaveBeenCalledOnce() + const finalPauseCall = messageStore.updateAssistantContent.mock.calls.findLast( + (call) => typeof call[2] === 'string' + ) + expect(finalPauseCall).toBeDefined() + expect(JSON.parse(finalPauseCall?.[2])).toMatchObject({ + providerRounds: 1, + toolCalls: 1, + runOutcome: 'paused', + runStopReason: 'interaction' + }) + }) + + it('enforces the global tool-call cap after a post-call permission pause', async () => { + const pausedToolPresenter = createPostCallPermissionToolPresenter() + + const pausedResult = await processStream( + createParams({ + coreStream: createToolRoundStream('action'), + toolExecution: createToolExecutionPort(pausedToolPresenter), + tools: [makeTool('action')], + permissionMode: 'default', + initialAccounting: { providerRounds: 0, toolCalls: 127 } + }) + ) + + expect(pausedResult).toMatchObject({ status: 'paused' }) + const finalPauseCall = messageStore.updateAssistantContent.mock.calls.findLast( + (call) => typeof call[2] === 'string' + ) + expect(finalPauseCall).toBeDefined() + const pausedMetadata = JSON.parse(finalPauseCall?.[2]) + expect(pausedMetadata).toMatchObject({ + toolCalls: 128, + runOutcome: 'paused', + runStopReason: 'interaction' + }) + + const resumedToolPresenter = createMockToolPresenter({ action: 'should not run' }) + const resumedResult = await processStream( + createParams({ + coreStream: createToolRoundStream('action'), + toolExecution: createToolExecutionPort(resumedToolPresenter), + tools: [makeTool('action')], + initialAccounting: pausedMetadata + }) + ) + + expect(resumedResult).toMatchObject({ status: 'completed', stopReason: 'max_tool_calls' }) + expect(resumedToolPresenter.callTool).not.toHaveBeenCalled() + expect( + JSON.parse(messageStore.finalizeAssistantMessage.mock.calls.at(-1)?.[2]) + ).toMatchObject({ + toolCalls: 128, + runOutcome: 'completed', + runStopReason: 'max_tool_calls' + }) + }) + it('settles a thrown provider error without a tool Tape snapshot', async () => { const order: string[] = [] observeCommitOrder(order) @@ -605,7 +754,7 @@ describe('processStream', () => { }) ) - expect(result).toMatchObject({ status: 'completed', stopReason: 'complete' }) + expect(result).toMatchObject({ status: 'completed', stopReason: 'max_tool_calls' }) expect(order).toEqual([ 'renderer:update', 'message:update', @@ -682,6 +831,57 @@ describe('processStream', () => { expect(tapeRecorder.appendToolFact).toHaveBeenCalledTimes(2) }) + it('keeps a tool-local AbortError as a tool failure while the run remains active', async () => { + let providerRound = 0 + const coreStream = vi.fn(() => { + providerRound += 1 + if (providerRound === 1) { + return (async function* () { + yield { + type: 'tool_call_start', + tool_call_id: 'tc1', + tool_call_name: 'action' + } as LLMCoreStreamEvent + yield { + type: 'tool_call_end', + tool_call_id: 'tc1', + tool_call_arguments_complete: '{}' + } as LLMCoreStreamEvent + yield { type: 'stop', stop_reason: 'tool_use' } as LLMCoreStreamEvent + })() + } + return (async function* () { + yield { type: 'text', content: 'Recovered' } as LLMCoreStreamEvent + yield { type: 'stop', stop_reason: 'complete' } as LLMCoreStreamEvent + })() + }) as unknown as ProcessParams['coreStream'] + const timeoutError = new Error('Model request timed out') + timeoutError.name = 'AbortError' + const toolPresenter = createMockToolPresenter() + ;(toolPresenter.callTool as ReturnType).mockRejectedValueOnce(timeoutError) + + const result = await processStream( + createParams({ + coreStream, + toolExecution: createToolExecutionPort(toolPresenter), + tools: [makeTool('action')] + }) + ) + + expect(result).toMatchObject({ status: 'completed', stopReason: 'complete' }) + expect(messageStore.setMessageError).not.toHaveBeenCalled() + expect(messageStore.finalizeAssistantMessage.mock.calls.at(-1)?.[1]).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + type: 'tool_call', + status: 'error', + tool_call: expect.objectContaining({ response: 'Error: Model request timed out' }) + }) + ]) + ) + expect(tapeRecorder.appendToolFact).toHaveBeenCalledTimes(2) + }) + it('persists the completed batch before settling for pending input', async () => { const order: string[] = [] observeCommitOrder(order) @@ -820,10 +1020,20 @@ describe('processStream', () => { await promise }) - it('treats AbortError thrown before the first event as aborted without writing an error block', async () => { + it('persists usage and aborted metadata when the provider throws AbortError', async () => { const abortError = new Error('Aborted') abortError.name = 'AbortError' const coreStream = vi.fn(async function* () { + yield { + type: 'usage', + usage: { + prompt_tokens: 11, + completion_tokens: 7, + total_tokens: 18, + cached_tokens: 3, + cache_write_tokens: 2 + } + } as LLMCoreStreamEvent throw abortError }) as unknown as ProcessParams['coreStream'] @@ -837,12 +1047,300 @@ describe('processStream', () => { stopReason: 'user_stop', errorMessage: 'common.error.userCanceledGeneration' }) - expect(messageStore.setMessageError).not.toHaveBeenCalled() + expect(result.usage).toEqual({ + inputTokens: 11, + outputTokens: 7, + totalTokens: 18, + cachedInputTokens: 3, + cacheWriteInputTokens: 2 + }) + expect(messageStore.setMessageError).toHaveBeenCalledOnce() + const metadata = JSON.parse(messageStore.setMessageError.mock.calls.at(-1)?.[2]) + expect(metadata).toMatchObject({ + runOutcome: 'aborted', + runStopReason: 'user_stop', + inputTokens: 11, + outputTokens: 7, + totalTokens: 18, + cachedInputTokens: 3, + cacheWriteInputTokens: 2, + providerRounds: 1, + toolCalls: 0 + }) expect(messageStore.finalizeAssistantMessage).not.toHaveBeenCalled() - expect(publishDeepchatEventMock).not.toHaveBeenCalledWith( - 'chat.stream.failed', - expect.anything() + expectDeepchatEvent('chat.stream.failed', { + requestId: 'req-1', + sessionId: 's1', + messageId: 'm1', + error: 'common.error.userCanceledGeneration' + }) + }) + + it('accounts for a tool that started before result normalization was aborted', async () => { + const abortController = new AbortController() + const coreStream = vi.fn(async function* () { + yield { + type: 'tool_call_start', + tool_call_id: 'tc-aborted', + tool_call_name: 'action' + } as LLMCoreStreamEvent + yield { + type: 'tool_call_end', + tool_call_id: 'tc-aborted', + tool_call_arguments_complete: '{}' + } as LLMCoreStreamEvent + yield { type: 'stop', stop_reason: 'tool_use' } as LLMCoreStreamEvent + }) as unknown as ProcessParams['coreStream'] + const toolPresenter = createMockToolPresenter({ action: 'raw result' }) + const toolResults = createToolResultPort({ + outputGuard: new ToolOutputGuard(), + normalize: async () => { + abortController.abort() + return 'normalized result' + } + }) + + const result = await processStream( + createParams({ + abortController, + coreStream, + toolExecution: createToolExecutionPort(toolPresenter), + toolResults, + tools: [makeTool('action')] + }) ) + + expect(result).toMatchObject({ status: 'aborted', stopReason: 'user_stop' }) + expect(toolPresenter.callTool).toHaveBeenCalledOnce() + expect(JSON.parse(messageStore.setMessageError.mock.calls.at(-1)?.[2])).toMatchObject({ + providerRounds: 1, + toolCalls: 1, + runOutcome: 'aborted', + runStopReason: 'user_stop' + }) + }) + + it('accumulates resumed accounting across provider and tool rounds', async () => { + let callCount = 0 + const coreStream = vi.fn(function () { + callCount++ + if (callCount === 1) { + return (async function* () { + yield { + type: 'usage', + usage: { + prompt_tokens: 2, + completion_tokens: 3, + total_tokens: 5, + cached_tokens: 1, + cache_write_tokens: 2 + } + } as LLMCoreStreamEvent + yield { + type: 'tool_call_start', + tool_call_id: 'tc-resumed', + tool_call_name: 'action' + } as LLMCoreStreamEvent + yield { + type: 'tool_call_end', + tool_call_id: 'tc-resumed', + tool_call_arguments_complete: '{}' + } as LLMCoreStreamEvent + yield { type: 'stop', stop_reason: 'tool_use' } as LLMCoreStreamEvent + })() + } + + return (async function* () { + yield { + type: 'usage', + usage: { + prompt_tokens: 4, + completion_tokens: 5, + total_tokens: 9, + cached_tokens: 2, + cache_write_tokens: 3 + } + } as LLMCoreStreamEvent + yield { type: 'text', content: 'Done' } as LLMCoreStreamEvent + yield { type: 'stop', stop_reason: 'complete' } as LLMCoreStreamEvent + })() + }) as unknown as ProcessParams['coreStream'] + const toolPresenter = createMockToolPresenter({ action: 'ok' }) + + const result = await processStream( + createParams({ + coreStream, + toolExecution: createToolExecutionPort(toolPresenter), + tools: [makeTool('action')], + initialAccounting: { + inputTokens: 10, + outputTokens: 20, + totalTokens: 30, + cachedInputTokens: 3, + cacheWriteInputTokens: 4, + providerRounds: 2, + toolCalls: 3, + generationTime: 1000, + firstTokenTime: 100 + } + }) + ) + + expect(result).toMatchObject({ + status: 'completed', + usage: { + inputTokens: 16, + outputTokens: 28, + totalTokens: 44, + cachedInputTokens: 6, + cacheWriteInputTokens: 9 + } + }) + expect(toolPresenter.callTool).toHaveBeenCalledOnce() + const metadata = JSON.parse(messageStore.finalizeAssistantMessage.mock.calls.at(-1)?.[2]) + expect(metadata).toMatchObject({ + inputTokens: 16, + outputTokens: 28, + totalTokens: 44, + cachedInputTokens: 6, + cacheWriteInputTokens: 9, + providerRounds: 4, + toolCalls: 4, + generationTime: 1000, + firstTokenTime: 100, + tokensPerSecond: 28, + runOutcome: 'completed', + runStopReason: 'complete' + }) + }) + + it('caps provider attempts reported by coreStream before starting another request', async () => { + const providerAttempts = vi.fn() + const coreStream = vi.fn(function (...args: Parameters) { + const onProviderRequestStart = args[6] + onProviderRequestStart?.() + providerAttempts() + onProviderRequestStart?.() + providerAttempts() + return (async function* () { + yield { type: 'stop', stop_reason: 'complete' } as LLMCoreStreamEvent + })() + }) as unknown as ProcessParams['coreStream'] + + const result = await processStream( + createParams({ + coreStream, + coreStreamReportsProviderStart: true, + maxProviderRounds: 1 + }) + ) + + expect(result).toMatchObject({ + status: 'error', + stopReason: 'max_turns', + errorMessage: 'Maximum agent turns exceeded (1).' + }) + expect(coreStream).toHaveBeenCalledOnce() + expect(providerAttempts).toHaveBeenCalledOnce() + const metadata = JSON.parse(messageStore.setMessageError.mock.calls.at(-1)?.[2]) + expect(metadata).toMatchObject({ + providerRounds: 1, + toolCalls: 0, + runOutcome: 'error', + runStopReason: 'max_turns' + }) + }) + + it('does not enter coreStream when resumed provider accounting already reached the cap', async () => { + const coreStream = vi.fn(async function* () { + yield { type: 'stop', stop_reason: 'complete' } as LLMCoreStreamEvent + }) as unknown as ProcessParams['coreStream'] + + const result = await processStream( + createParams({ + coreStream, + coreStreamReportsProviderStart: true, + initialAccounting: { providerRounds: 1, toolCalls: 0 }, + maxProviderRounds: 1 + }) + ) + + expect(result).toMatchObject({ + status: 'error', + stopReason: 'max_turns', + errorMessage: 'Maximum agent turns exceeded (1).' + }) + expect(coreStream).not.toHaveBeenCalled() + expect(JSON.parse(messageStore.setMessageError.mock.calls.at(-1)?.[2])).toMatchObject({ + providerRounds: 1, + toolCalls: 0, + runOutcome: 'error', + runStopReason: 'max_turns' + }) + }) + + it('restores the persisted provider-round cap after an interaction pause', async () => { + const pauseCoreStream = vi.fn(async function* () { + yield { + type: 'tool_call_start', + tool_call_id: 'question-provider-cap', + tool_call_name: 'deepchat_question' + } as LLMCoreStreamEvent + yield { + type: 'tool_call_end', + tool_call_id: 'question-provider-cap', + tool_call_arguments_complete: JSON.stringify({ + question: 'Continue?', + options: [{ label: 'Yes' }, { label: 'No' }] + }) + } as LLMCoreStreamEvent + yield { type: 'stop', stop_reason: 'tool_use' } as LLMCoreStreamEvent + }) as unknown as ProcessParams['coreStream'] + + const pausedResult = await processStream( + createParams({ + coreStream: pauseCoreStream, + tools: [makeTool('deepchat_question')], + maxProviderRounds: 1 + }) + ) + + expect(pausedResult).toMatchObject({ status: 'paused' }) + const finalPauseCall = messageStore.updateAssistantContent.mock.calls.findLast( + (call) => typeof call[2] === 'string' + ) + expect(finalPauseCall).toBeDefined() + const pausedMetadata = JSON.parse(finalPauseCall?.[2]) + expect(pausedMetadata).toMatchObject({ + maxProviderRounds: 1, + providerRounds: 1, + runOutcome: 'paused', + runStopReason: 'interaction' + }) + + const resumedCoreStream = vi.fn(async function* () { + yield { type: 'stop', stop_reason: 'complete' } as LLMCoreStreamEvent + }) as unknown as ProcessParams['coreStream'] + const resumedResult = await processStream( + createParams({ + coreStream: resumedCoreStream, + coreStreamReportsProviderStart: true, + initialAccounting: pausedMetadata + }) + ) + + expect(resumedResult).toMatchObject({ + status: 'error', + stopReason: 'max_turns', + errorMessage: 'Maximum agent turns exceeded (1).' + }) + expect(resumedCoreStream).not.toHaveBeenCalled() + expect(JSON.parse(messageStore.setMessageError.mock.calls.at(-1)?.[2])).toMatchObject({ + maxProviderRounds: 1, + providerRounds: 1, + runOutcome: 'error', + runStopReason: 'max_turns' + }) }) it('single tool call → loop once, finalize', async () => { @@ -1775,8 +2273,24 @@ describe('processStream', () => { errorMessage: 'common.error.userCanceledGeneration' }) expect(messageStore.updateAssistantContent).not.toHaveBeenCalled() - expect(messageStore.setMessageError).not.toHaveBeenCalled() + expect(messageStore.setMessageError).toHaveBeenCalledWith( + 'm1', + expect.arrayContaining([ + expect.objectContaining({ + type: 'error', + content: 'common.error.userCanceledGeneration', + status: 'error' + }) + ]), + expect.any(String) + ) expect(messageStore.finalizeAssistantMessage).not.toHaveBeenCalled() + expect(JSON.parse(messageStore.setMessageError.mock.calls.at(-1)?.[2])).toMatchObject({ + runOutcome: 'aborted', + runStopReason: 'user_stop', + providerRounds: 1, + toolCalls: 0 + }) expectDeepchatEvent('chat.plan.updated', { sessionId: 's1', messageId: 'm1', @@ -1805,18 +2319,23 @@ describe('processStream', () => { stopReason: 'user_stop', errorMessage: 'common.error.userCanceledGeneration' }) - expect(messageStore.setMessageError).not.toHaveBeenCalled() - expect(messageStore.finalizeAssistantMessage).not.toHaveBeenCalled() - expect(messageStore.updateAssistantContent).toHaveBeenLastCalledWith( + expect(messageStore.setMessageError).toHaveBeenCalledWith( 'm1', expect.arrayContaining([ expect.objectContaining({ type: 'content', content: 'Partial answer', status: 'success' + }), + expect.objectContaining({ + type: 'error', + content: 'common.error.userCanceledGeneration', + status: 'error' }) - ]) + ]), + expect.any(String) ) + expect(messageStore.finalizeAssistantMessage).not.toHaveBeenCalled() expectDeepchatEvent('chat.plan.updated', { sessionId: 's1', messageId: 'm1', @@ -1973,15 +2492,16 @@ describe('processStream', () => { const promise = processStream(params) await vi.runAllTimersAsync() - await promise + const result = await promise - // Error event is accumulated into blocks, stop_reason becomes 'error'. - // Since stop_reason != 'tool_use', it breaks out and calls finalize. - // The error block was already accumulated by the accumulator. - // finalize marks remaining pending blocks as success. - // This matches the v2 behavior where error events from the stream - // still lead to finalization (blocks contain the error block). - expect(messageStore.finalizeAssistantMessage).toHaveBeenCalled() + // Generic provider error events finalize as errors (not success-shaped complete). + expect(result).toMatchObject({ + status: 'error', + stopReason: 'provider_error', + errorMessage: 'Rate limit exceeded' + }) + expect(messageStore.setMessageError).toHaveBeenCalled() + expect(messageStore.finalizeAssistantMessage).not.toHaveBeenCalled() }) it('context window error event is finalized as an error', async () => { diff --git a/test/main/presenter/agentSessionPresenter/integration.test.ts b/test/main/presenter/agentSessionPresenter/integration.test.ts index 2354aa7c2..ee4abf0bb 100644 --- a/test/main/presenter/agentSessionPresenter/integration.test.ts +++ b/test/main/presenter/agentSessionPresenter/integration.test.ts @@ -362,6 +362,13 @@ function createMockSqlitePresenter() { const msg = messagesStore.get(id) if (msg) msg.status = status }), + updateMetadata: vi.fn((id: string, metadata: string) => { + const msg = messagesStore.get(id) + if (msg) { + msg.metadata = metadata + msg.updated_at = Date.now() + } + }), getBySession: vi.fn((sessionId: string) => { return messagesList .filter((m) => m.session_id === sessionId) @@ -1017,7 +1024,7 @@ describe('Integration: multi-turn context', () => { modelId: 'gpt-4', generationSettings: { contextLength: 8192, maxTokens: 4096 } }) - await deepchatAgent.processMessage(sessionId, 'Hello', { maxProviderRounds: 1 }) + await deepchatAgent.processMessage(sessionId, 'Hello', { maxProviderRounds: 2 }) expect(providerInstance.coreStream).toHaveBeenCalledTimes(2) expect(observedRuns).toHaveLength(2) diff --git a/test/main/presenter/llmProviderPresenter.test.ts b/test/main/presenter/llmProviderPresenter.test.ts index 4b8f2589c..9bc692311 100644 --- a/test/main/presenter/llmProviderPresenter.test.ts +++ b/test/main/presenter/llmProviderPresenter.test.ts @@ -361,6 +361,81 @@ describe('LLMProviderPresenter Integration Tests', () => { expect(response.length).toBeGreaterThan(0) }, 15000) + it('observes a completion failure that arrives after standalone cancellation', async () => { + let rejectCompletion!: (reason?: unknown) => void + const completion = new Promise((_, reject) => { + rejectCompletion = reject + }) + const provider = llmProviderPresenter.getProviderInstance('mock-openai-api') + const completionsSpy = vi.spyOn(provider, 'completions').mockReturnValue(completion) + const abortController = new AbortController() + const lateError = new Error('late standalone completion failure') + const unhandled = vi.fn() + + try { + const generating = llmProviderPresenter.generateCompletionStandalone( + 'mock-openai-api', + [{ role: 'user', content: 'cancel me' }], + 'mock-gpt-thinking', + undefined, + undefined, + { signal: abortController.signal } + ) + abortController.abort() + + await expect(generating).rejects.toMatchObject({ name: 'AbortError' }) + + process.on('unhandledRejection', unhandled) + try { + rejectCompletion(lateError) + await new Promise((resolve) => setImmediate(resolve)) + await new Promise((resolve) => setImmediate(resolve)) + expect(unhandled.mock.calls.some(([reason]) => reason === lateError)).toBe(false) + } finally { + process.off('unhandledRejection', unhandled) + } + } finally { + completionsSpy.mockRestore() + } + }) + + it('consumes an iterator teardown failure during standalone image cancellation', async () => { + const lateError = new Error('image iterator teardown failed') + const stream = { + next: vi.fn(() => new Promise>(() => undefined)), + return: vi.fn().mockRejectedValue(lateError), + [Symbol.asyncIterator]() { + return this + } + } + const provider = llmProviderPresenter.getProviderInstance('mock-openai-api') + const coreStreamSpy = vi.spyOn(provider, 'coreStream').mockReturnValue(stream as any) + const abortController = new AbortController() + const unhandled = vi.fn() + + process.on('unhandledRejection', unhandled) + try { + const generating = llmProviderPresenter.generateImageStandalone( + 'mock-openai-api', + 'cancel me', + 'mock-gpt-thinking', + undefined, + { signal: abortController.signal } + ) + await vi.waitFor(() => expect(coreStreamSpy).toHaveBeenCalledOnce()) + abortController.abort() + + await expect(generating).rejects.toMatchObject({ name: 'AbortError' }) + await new Promise((resolve) => setImmediate(resolve)) + await new Promise((resolve) => setImmediate(resolve)) + expect(stream.return).toHaveBeenCalledOnce() + expect(unhandled.mock.calls.some(([reason]) => reason === lateError)).toBe(false) + } finally { + process.off('unhandledRejection', unhandled) + coreStreamSpy.mockRestore() + } + }) + it('falls back to completion transcription when audio endpoint is unsupported', async () => { vi.stubGlobal( 'fetch', diff --git a/test/main/presenter/mcpClient.test.ts b/test/main/presenter/mcpClient.test.ts index b0546581d..9f47ad375 100644 --- a/test/main/presenter/mcpClient.test.ts +++ b/test/main/presenter/mcpClient.test.ts @@ -630,6 +630,148 @@ describe('McpClient Runtime Command Processing Tests', () => { }) }) + describe('Tool cancellation', () => { + const deferred = () => { + let resolve!: (value: T) => void + let reject!: (reason?: unknown) => void + const promise = new Promise((promiseResolve, promiseReject) => { + resolve = promiseResolve + reject = promiseReject + }) + return { promise, resolve, reject } + } + + const configureConnectedClient = ( + sdkCallTool: ReturnType, + cachedTools: unknown[] = [{ name: 'cached-tool' }] + ) => { + const client = new McpClient('cancellable-server', { type: 'stdio' }) + ;(client as any).client = { callTool: sdkCallTool } + ;(client as any).isConnected = true + ;(client as any).cachedTools = cachedTools + return client + } + + it('rejects a pre-aborted call without invoking the SDK or session recovery', async () => { + const sdkCallTool = vi.fn() + const cachedTools = [{ name: 'cached-tool' }] + const client = configureConnectedClient(sdkCallTool, cachedTools) + const recoverySpy = vi.spyOn(client as any, 'checkAndHandleSessionError') + const abortController = new AbortController() + abortController.abort() + + await expect( + client.callTool('echo', {}, { signal: abortController.signal }) + ).rejects.toMatchObject({ name: 'AbortError' }) + + expect(sdkCallTool).not.toHaveBeenCalled() + expect(recoverySpy).not.toHaveBeenCalled() + expect((client as any).cachedTools).toBe(cachedTools) + }) + + it('passes the signal to the SDK and skips recovery when an in-flight call aborts', async () => { + const sdkCallTool = vi.fn( + (_request: unknown, _schema: unknown, requestOptions?: { signal?: AbortSignal }) => + new Promise((_, reject) => { + const signal = requestOptions?.signal + if (!signal) { + reject(new Error('Missing abort signal')) + return + } + if (signal.aborted) { + reject(signal.reason) + return + } + signal.addEventListener('abort', () => reject(signal.reason), { once: true }) + }) + ) + const cachedTools = [{ name: 'cached-tool' }] + const client = configureConnectedClient(sdkCallTool, cachedTools) + const recoverySpy = vi.spyOn(client as any, 'checkAndHandleSessionError') + const abortController = new AbortController() + + const callPromise = client.callTool('echo', { value: 1 }, { signal: abortController.signal }) + await vi.waitFor(() => { + expect(sdkCallTool).toHaveBeenCalledWith( + { name: 'echo', arguments: { value: 1 } }, + undefined, + { signal: abortController.signal } + ) + }) + + abortController.abort() + + await expect(callPromise).rejects.toMatchObject({ name: 'AbortError' }) + expect(recoverySpy).not.toHaveBeenCalled() + expect((client as any).cachedTools).toBe(cachedTools) + }) + + it('rejects promptly when cancellation lands during connection setup', async () => { + const sdkCallTool = vi.fn() + const client = configureConnectedClient(sdkCallTool) + const connection = deferred() + const connect = vi.spyOn(client, 'connect').mockReturnValue(connection.promise) + ;(client as any).isConnected = false + const abortController = new AbortController() + + const callPromise = client.callTool('echo', {}, { signal: abortController.signal }) + await vi.waitFor(() => expect(connect).toHaveBeenCalledOnce()) + + abortController.abort() + + await expect(callPromise).rejects.toMatchObject({ name: 'AbortError' }) + expect(sdkCallTool).not.toHaveBeenCalled() + connection.resolve('connected') + }) + + it('observes a connection failure that arrives after setup synchronously cancels', async () => { + const sdkCallTool = vi.fn() + const client = configureConnectedClient(sdkCallTool) + const connection = deferred() + const abortController = new AbortController() + const lateError = new Error('late connection failure') + const unhandled = vi.fn() + const connect = vi.spyOn(client, 'connect').mockImplementation(() => { + abortController.abort() + return connection.promise + }) + ;(client as any).isConnected = false + + await expect( + client.callTool('echo', {}, { signal: abortController.signal }) + ).rejects.toMatchObject({ name: 'AbortError' }) + expect(connect).toHaveBeenCalledOnce() + expect(sdkCallTool).not.toHaveBeenCalled() + + process.on('unhandledRejection', unhandled) + try { + connection.reject(lateError) + await new Promise((resolve) => setImmediate(resolve)) + await new Promise((resolve) => setImmediate(resolve)) + expect(unhandled.mock.calls.some(([reason]) => reason === lateError)).toBe(false) + } finally { + process.off('unhandledRejection', unhandled) + } + }) + + it('preserves session recovery and cache invalidation for non-abort failures', async () => { + const sdkError = new Error('MCP request failed') + const sdkCallTool = vi.fn().mockRejectedValue(sdkError) + const client = configureConnectedClient(sdkCallTool) + const recoverySpy = vi + .spyOn(client as any, 'checkAndHandleSessionError') + .mockResolvedValue(undefined) + const consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => undefined) + + await expect(client.callTool('echo', {})).rejects.toBe(sdkError) + + expect(sdkCallTool).toHaveBeenCalledTimes(1) + expect(recoverySpy).toHaveBeenCalledWith(sdkError) + expect((client as any).cachedTools).toBeNull() + consoleErrorSpy.mockRestore() + }) + }) + describe('Sampling support', () => { it('should prepare sampling payload and chat messages from request params', () => { const client = new McpClient('server-one', { @@ -805,7 +947,8 @@ describe('McpClient Runtime Command Processing Tests', () => { ], 'model-42', undefined, - 256 + 256, + { signal: undefined, swallowErrors: false } ) expect(result).toEqual({ @@ -843,5 +986,88 @@ describe('McpClient Runtime Command Processing Tests', () => { expect(mockGenerateCompletionStandalone).not.toHaveBeenCalled() }) + + it('observes a late sampling decision failure after the request was cancelled', async () => { + let rejectDecision!: (reason?: unknown) => void + const decision = new Promise((_, reject) => { + rejectDecision = reject + }) + const client = new McpClient('code-reviewer', { type: 'stdio' }) + const abortController = new AbortController() + const lateError = new Error('late sampling decision failure') + const unhandled = vi.fn() + mockHandleSamplingRequest.mockReturnValue(decision) + mockCancelSamplingRequest.mockResolvedValue(undefined) + abortController.abort() + + await expect( + (client as any).handleSamplingCreateMessage( + { + params: { + messages: [{ role: 'user', content: { type: 'text', text: 'hello' } }] + } + }, + { requestId: 'rpc-cancelled', signal: abortController.signal } + ) + ).rejects.toMatchObject({ code: ErrorCode.RequestTimeout }) + expect(mockCancelSamplingRequest).toHaveBeenCalledWith('rpc-cancelled', 'cancelled by server') + + process.on('unhandledRejection', unhandled) + try { + rejectDecision(lateError) + await new Promise((resolve) => setImmediate(resolve)) + await new Promise((resolve) => setImmediate(resolve)) + expect(unhandled.mock.calls.some(([reason]) => reason === lateError)).toBe(false) + } finally { + process.off('unhandledRejection', unhandled) + } + }) + + it('forwards cancellation through sampling generation without wrapping AbortError', async () => { + const client = new McpClient('code-reviewer', { type: 'stdio' }) + const abortController = new AbortController() + mockHandleSamplingRequest.mockResolvedValue({ + requestId: 'rpc-generation-cancelled', + approved: true, + providerId: 'provider-1', + modelId: 'model-42' + }) + mockCancelSamplingRequest.mockResolvedValue(undefined) + mockGenerateCompletionStandalone.mockImplementation( + (...args: unknown[]) => + new Promise((_, reject) => { + const options = args[5] as { signal?: AbortSignal } | undefined + options?.signal?.addEventListener('abort', () => reject(options.signal?.reason), { + once: true + }) + }) + ) + + const generating = (client as any).handleSamplingCreateMessage( + { + params: { + messages: [{ role: 'user', content: { type: 'text', text: 'hello' } }] + } + }, + { requestId: 'rpc-generation-cancelled', signal: abortController.signal } + ) + await vi.waitFor(() => expect(mockGenerateCompletionStandalone).toHaveBeenCalledOnce()) + + abortController.abort() + + await expect(generating).rejects.toMatchObject({ name: 'AbortError' }) + expect(mockGenerateCompletionStandalone).toHaveBeenCalledWith( + 'provider-1', + expect.any(Array), + 'model-42', + undefined, + undefined, + { signal: abortController.signal, swallowErrors: false } + ) + expect(mockCancelSamplingRequest).toHaveBeenCalledWith( + 'rpc-generation-cancelled', + 'cancelled by server' + ) + }) }) }) diff --git a/test/main/presenter/mcpPresenter/toolManager.test.ts b/test/main/presenter/mcpPresenter/toolManager.test.ts index 2a952dc7a..375a2eaa4 100644 --- a/test/main/presenter/mcpPresenter/toolManager.test.ts +++ b/test/main/presenter/mcpPresenter/toolManager.test.ts @@ -32,6 +32,16 @@ vi.mock('@/presenter', () => ({ import { ToolManager } from '../../../../src/main/presenter/mcpPresenter/toolManager' +function deferred() { + let resolve!: (value: T) => void + let reject!: (reason?: unknown) => void + const promise = new Promise((promiseResolve, promiseReject) => { + resolve = promiseResolve + reject = promiseReject + }) + return { promise, resolve, reject } +} + describe('ToolManager', () => { let warnSpy: ReturnType @@ -366,6 +376,308 @@ describe('ToolManager', () => { ).toBe(false) }) + it('forwards the caller abort signal to the selected MCP client', async () => { + const client = createClient('open-server') + client.callTool.mockImplementation( + (_name: string, _args: Record, options?: { signal?: AbortSignal }) => + new Promise((_, reject) => { + const signal = options?.signal + if (!signal) { + reject(new Error('Missing abort signal')) + return + } + signal.addEventListener('abort', () => reject(signal.reason), { once: true }) + }) + ) + const configPresenter = createConfigPresenter('open-server') + const manager = new ToolManager( + configPresenter as never, + createServerManager([client]) as never + ) + const abortController = new AbortController() + + const callPromise = manager.callTool( + { + id: 'tool-cancellable', + type: 'function', + function: { + name: 'echo', + arguments: '{}' + }, + conversationId: 'conv-cancellable', + providerId: 'openai' + }, + { signal: abortController.signal } + ) + + await vi.waitFor(() => { + expect(client.callTool).toHaveBeenCalledWith('echo', {}, { signal: abortController.signal }) + }) + abortController.abort() + + await expect(callPromise).rejects.toMatchObject({ name: 'AbortError' }) + }) + + it('rejects promptly when cancellation lands during tool-definition refresh', async () => { + const client = createClient('open-server') + const configPresenter = createConfigPresenter('open-server') + const manager = new ToolManager( + configPresenter as never, + createServerManager([client]) as never + ) + const definitions = deferred>>() + const loadDefinitions = vi + .spyOn(manager, 'getAllToolDefinitions') + .mockReturnValue(definitions.promise) + const abortController = new AbortController() + + const callPromise = manager.callTool( + { + id: 'tool-preflight-cancel', + type: 'function', + function: { name: 'echo', arguments: '{}' }, + conversationId: 'conv-preflight-cancel', + providerId: 'openai' + }, + { signal: abortController.signal } + ) + await vi.waitFor(() => expect(loadDefinitions).toHaveBeenCalledOnce()) + + abortController.abort() + + await expect(callPromise).rejects.toMatchObject({ name: 'AbortError' }) + expect(client.callTool).not.toHaveBeenCalled() + definitions.resolve([]) + }) + + it('does not commit tool definitions after their refresh is cancelled', async () => { + const client = createClient('stale-server') + const tools = deferred>>() + client.listTools.mockReturnValue(tools.promise) + const manager = new ToolManager( + createConfigPresenter('stale-server') as never, + createServerManager([client]) as never + ) + const abortController = new AbortController() + + const refresh = manager.getAllToolDefinitions(undefined, { + signal: abortController.signal + }) + await vi.waitFor(() => expect(client.listTools).toHaveBeenCalledOnce()) + + abortController.abort() + + await expect(refresh).rejects.toMatchObject({ name: 'AbortError' }) + tools.resolve([ + { + name: 'stale_tool', + description: 'Stale tool', + inputSchema: { properties: {}, required: [] } + } + ]) + await new Promise((resolve) => setImmediate(resolve)) + + expect((manager as any).cachedToolDefinitions).toBeNull() + expect((manager as any).toolNameToTargetMap).toBeNull() + }) + + it('discards a refresh superseded by a configuration cache clear', async () => { + const staleClient = createClient('stale-server') + const currentClient = createClient('current-server', [ + { + name: 'current_tool', + description: 'Current tool', + inputSchema: { properties: {}, required: [] } + } + ]) + const staleTools = deferred>>() + staleClient.listTools.mockReturnValueOnce(staleTools.promise).mockResolvedValueOnce([ + { + name: 'stale_tool', + description: 'Stale tool', + inputSchema: { properties: {}, required: [] } + } + ]) + const serverManager = createServerManager([staleClient]) + const manager = new ToolManager( + createConfigPresenter('current-server') as never, + serverManager as never + ) + + const refresh = manager.getAllToolDefinitions() + await vi.waitFor(() => expect(staleClient.listTools).toHaveBeenCalledOnce()) + ;(manager as any).handleConfigChange() + serverManager.getRunningClients.mockResolvedValue([currentClient]) + staleTools.resolve([ + { + name: 'stale_tool', + description: 'Stale tool', + inputSchema: { properties: {}, required: [] } + } + ]) + + const definitions = await refresh + + expect(definitions.map((definition) => definition.function.name)).toEqual(['current_tool']) + expect((manager as any).toolNameToTargetMap.has('current_tool')).toBe(true) + expect((manager as any).toolNameToTargetMap.has('stale_tool')).toBe(false) + }) + + it('wakes refresh waiters when configuration invalidates a blocked refresh', async () => { + const staleClient = createClient('stale-server') + const currentClient = createClient('current-server', [ + { + name: 'current_tool', + description: 'Current tool', + inputSchema: { properties: {}, required: [] } + } + ]) + const staleTools = deferred>>() + staleClient.listTools.mockReturnValue(staleTools.promise) + const serverManager = createServerManager([staleClient]) + const manager = new ToolManager( + createConfigPresenter('current-server') as never, + serverManager as never + ) + + const blockedRefresh = manager.getAllToolDefinitions() + await vi.waitFor(() => expect(staleClient.listTools).toHaveBeenCalledOnce()) + const waitingRefresh = manager.getAllToolDefinitions() + + serverManager.getRunningClients.mockResolvedValue([currentClient]) + ;(manager as any).handleConfigChange() + + await expect(waitingRefresh).resolves.toEqual([ + expect.objectContaining({ function: expect.objectContaining({ name: 'current_tool' }) }) + ]) + expect(currentClient.listTools).toHaveBeenCalled() + + staleTools.resolve([]) + await expect(blockedRefresh).resolves.toEqual([ + expect.objectContaining({ function: expect.objectContaining({ name: 'current_tool' }) }) + ]) + }) + + it('observes a definition failure after refresh synchronously cancels the call', async () => { + const client = createClient('open-server') + const manager = new ToolManager( + createConfigPresenter('open-server') as never, + createServerManager([client]) as never + ) + const definitions = deferred>>() + const abortController = new AbortController() + const lateError = new Error('late definition failure') + const unhandled = vi.fn() + vi.spyOn(manager, 'getAllToolDefinitions').mockImplementation(() => { + abortController.abort() + return definitions.promise + }) + + await expect( + manager.callTool( + { + id: 'tool-sync-cancel', + type: 'function', + function: { name: 'echo', arguments: '{}' }, + providerId: 'openai' + }, + { signal: abortController.signal } + ) + ).rejects.toMatchObject({ name: 'AbortError' }) + expect(client.callTool).not.toHaveBeenCalled() + + process.on('unhandledRejection', unhandled) + try { + definitions.reject(lateError) + await new Promise((resolve) => setImmediate(resolve)) + await new Promise((resolve) => setImmediate(resolve)) + expect(unhandled.mock.calls.some(([reason]) => reason === lateError)).toBe(false) + } finally { + process.off('unhandledRejection', unhandled) + } + }) + + it('does not start tool-definition refresh for a pre-cancelled call', async () => { + const client = createClient('open-server') + const manager = new ToolManager( + createConfigPresenter('open-server') as never, + createServerManager([client]) as never + ) + const loadDefinitions = vi.spyOn(manager, 'getAllToolDefinitions') + const abortController = new AbortController() + abortController.abort() + + await expect( + manager.callTool( + { + id: 'tool-pre-cancelled', + type: 'function', + function: { name: 'echo', arguments: '{}' }, + providerId: 'openai' + }, + { signal: abortController.signal } + ) + ).rejects.toMatchObject({ name: 'AbortError' }) + expect(loadDefinitions).not.toHaveBeenCalled() + expect(client.callTool).not.toHaveBeenCalled() + }) + + it('rejects permission pre-check promptly during tool-definition refresh', async () => { + const client = createClient('open-server') + const manager = new ToolManager( + createConfigPresenter('open-server') as never, + createServerManager([client]) as never + ) + const definitions = deferred>>() + const loadDefinitions = vi + .spyOn(manager, 'getAllToolDefinitions') + .mockReturnValue(definitions.promise) + const abortController = new AbortController() + + const checking = manager.preCheckToolPermission( + { + id: 'permission-preflight-cancel', + type: 'function', + function: { name: 'echo', arguments: '{}' } + }, + { signal: abortController.signal } + ) + await vi.waitFor(() => expect(loadDefinitions).toHaveBeenCalledOnce()) + + abortController.abort() + + await expect(checking).rejects.toMatchObject({ name: 'AbortError' }) + definitions.resolve([]) + }) + + it('rejects permission pre-check promptly while server config is loading', async () => { + const client = createClient('open-server') + const configPresenter = createConfigPresenter('open-server') + const manager = new ToolManager( + configPresenter as never, + createServerManager([client]) as never + ) + await manager.getAllToolDefinitions() + const servers = deferred>>() + configPresenter.getMcpServers.mockReturnValue(servers.promise) + const abortController = new AbortController() + + const checking = manager.preCheckToolPermission( + { + id: 'permission-config-cancel', + type: 'function', + function: { name: 'echo', arguments: '{}' } + }, + { signal: abortController.signal } + ) + await vi.waitFor(() => expect(configPresenter.getMcpServers).toHaveBeenCalledOnce()) + + abortController.abort() + + await expect(checking).rejects.toMatchObject({ name: 'AbortError' }) + servers.resolve({ 'open-server': { autoApprove: ['all'] } }) + }) + it('skips ACP selection gating for non-ACP sessions', async () => { const client = createClient('open-server') const configPresenter = createConfigPresenter('open-server') @@ -459,6 +771,45 @@ describe('ToolManager', () => { expect(client.callTool).toHaveBeenCalledWith('list_apps', {}) }) + it('forwards and preserves cancellation from the CUA Windows preflight helper', async () => { + const client = createClient('cua-driver', [], { + source: 'plugin', + ownerPluginId: 'com.deepchat.plugins.cua' + }) + client.callTool.mockImplementation( + (_name: string, _args: Record, options?: { signal?: AbortSignal }) => + new Promise((_, reject) => { + options?.signal?.addEventListener('abort', () => reject(options.signal?.reason), { + once: true + }) + }) + ) + const manager = new ToolManager( + createConfigPresenter('cua-driver') as never, + createServerManager([client]) as never + ) + const abortController = new AbortController() + + const preparing = (manager as any).prepareCuaWindowsLaunchArgs( + client, + { bundle_id: 'com.apple.TextEdit' }, + abortController.signal + ) + await vi.waitFor(() => + expect(client.callTool).toHaveBeenCalledWith( + 'list_apps', + {}, + { + signal: abortController.signal + } + ) + ) + + abortController.abort() + + await expect(preparing).rejects.toMatchObject({ name: 'AbortError' }) + }) + it('treats missing provider hint as a fallback to new session resolution', async () => { const client = createClient('open-server') const configPresenter = createConfigPresenter('open-server') diff --git a/test/main/presenter/openaiCodexAuth.test.ts b/test/main/presenter/openaiCodexAuth.test.ts index e36a1b4df..ada8f1ca6 100644 --- a/test/main/presenter/openaiCodexAuth.test.ts +++ b/test/main/presenter/openaiCodexAuth.test.ts @@ -6,10 +6,23 @@ import { OpenAICodexAuth } from '../../../src/main/presenter/openaiCodexAuth' import { OpenAICodexCredentialStore } from '../../../src/main/presenter/openaiCodexAuth/credentialStore' import { createOpenAICodexPkcePair } from '../../../src/main/presenter/openaiCodexAuth/pkce' +const { startOAuthLoopbackCallbackSessionMock } = vi.hoisted(() => ({ + startOAuthLoopbackCallbackSessionMock: vi.fn() +})) + vi.mock('@/routes/publishDeepchatEvent', () => ({ publishDeepchatEvent: vi.fn() })) +vi.mock('../../../src/main/presenter/oauthLoopbackCallback', async (importOriginal) => { + const actual = + await importOriginal() + return { + ...actual, + startOAuthLoopbackCallbackSession: startOAuthLoopbackCallbackSessionMock + } +}) + describe('OpenAI Codex auth', () => { let tempDir: string let files: Map @@ -26,6 +39,7 @@ describe('OpenAI Codex auth', () => { vi.mocked(fs.rmSync).mockImplementation((file) => { files.delete(String(file)) }) + startOAuthLoopbackCallbackSessionMock.mockReset() vi.mocked(shell.openExternal).mockClear() delete process.env.DEEPCHAT_OPENAI_CODEX_DISABLED }) @@ -153,6 +167,44 @@ describe('OpenAI Codex auth', () => { }) it('opens browser login externally and completes from a pasted callback URL', async () => { + startOAuthLoopbackCallbackSessionMock.mockImplementationOnce( + async (options: { expectedState: string; path: string }) => { + const callbackPath = options.path.startsWith('/') ? options.path : `/${options.path}` + const redirectUri = `http://localhost:43123${callbackPath}` + let resolveCallback!: (value: { code: string }) => void + let rejectCallback!: (error: Error) => void + const callbackPromise = new Promise<{ code: string }>((resolve, reject) => { + resolveCallback = resolve + rejectCallback = reject + }) + + return { + redirectUri, + waitForCallback: vi.fn(() => callbackPromise), + resolveCallbackUrl: vi.fn((rawUrl: string) => { + const callbackUrl = new URL(rawUrl) + const code = callbackUrl.searchParams.get('code') + const state = callbackUrl.searchParams.get('state') + if (!code || state !== options.expectedState) { + const error = new Error('Invalid OAuth callback') + rejectCallback(error) + return { kind: 'failure' as const, error, url: callbackUrl.toString() } + } + + const result = { + kind: 'success' as const, + code, + state, + url: callbackUrl.toString() + } + resolveCallback(result) + return result + }), + close: vi.fn() + } + } + ) + const store = new OpenAICodexCredentialStore(path.join(tempDir, 'browser.json')) const fetchMock = vi.fn().mockResolvedValue( new Response( diff --git a/test/main/presenter/toolPresenter/agentTools/subagentOrchestratorTool.test.ts b/test/main/presenter/toolPresenter/agentTools/subagentOrchestratorTool.test.ts index 82c74281d..79ffec18c 100644 --- a/test/main/presenter/toolPresenter/agentTools/subagentOrchestratorTool.test.ts +++ b/test/main/presenter/toolPresenter/agentTools/subagentOrchestratorTool.test.ts @@ -35,6 +35,53 @@ const buildSessionInfo = ( ...overrides }) +const buildRuntimePort = ( + parentSession: ConversationSessionInfo, + overrides: Record = {} +) => ({ + resolveConversationWorkdir: vi.fn().mockResolvedValue(parentSession.projectDir), + resolveConversationSessionInfo: vi.fn().mockResolvedValue(parentSession), + createSubagentSession: vi.fn().mockImplementation(async () => + buildSessionInfo({ + sessionId: 'child-session', + sessionKind: 'subagent', + parentSessionId: parentSession.sessionId, + subagentEnabled: false, + availableSubagentSlots: [] + }) + ), + sendConversationMessage: vi.fn().mockResolvedValue(undefined), + cancelConversation: vi.fn().mockResolvedValue(undefined), + subscribeDeepChatSessionUpdates: vi.fn(() => () => undefined), + mergeSubagentTape: vi.fn().mockResolvedValue(undefined), + discardSubagentTape: vi.fn().mockResolvedValue(undefined), + getSkillPresenter: vi.fn(() => ({})), + getYoBrowserToolHandler: vi.fn(() => ({})), + getFilePresenter: vi.fn(() => ({ + getMimeType: vi.fn(), + prepareFileCompletely: vi.fn() + })), + getLlmProviderPresenter: vi.fn(() => ({ + executeWithRateLimit: vi.fn().mockResolvedValue(undefined), + generateCompletionStandalone: vi.fn(), + generateImageStandalone: vi.fn() + })), + createSettingsWindow: vi.fn(), + sendToWindow: vi.fn(), + getApprovedFilePaths: vi.fn(() => []), + consumeSettingsApproval: vi.fn(() => false), + ...overrides +}) + +const createDeferredPromise = () => { + let resolve!: (value: T | PromiseLike) => void + const promise = new Promise((innerResolve) => { + resolve = innerResolve + }) + + return { promise, resolve } +} + describe('SubagentOrchestratorTool', () => { it('includes the parent session workdir in the child handoff', async () => { let listener: ((update: DeepChatInternalSessionUpdate) => void) | null = null @@ -137,9 +184,434 @@ describe('SubagentOrchestratorTool', () => { expect(handoffMessage).not.toContain('Review the delegated task.') expect(handoffMessage).not.toContain(parentSession.projectDir as string) expect(handoffMessage).not.toContain(childSession.projectDir as string) + expect(handoffMessage).toContain('## Result') + expect(handoffMessage).toContain('## Evidence') + expect(handoffMessage).toContain('## Changed Files') + expect(handoffMessage).toContain('## Validation') + expect(handoffMessage).toContain('## Unresolved') + expect(handoffMessage).toContain('Use `None` as the section content') + expect(handoffMessage).toContain('Additional Requirements:') + expect(handoffMessage).toContain('Return concise markdown findings.') expect(result.content).toContain('Inspect auth flow') }) + it('enforces and serializes a run deadline independently of wait timeout', async () => { + vi.useFakeTimers() + vi.setSystemTime(new Date('2026-07-13T00:00:00.000Z')) + + try { + const parentSession = buildSessionInfo() + const childSession = buildSessionInfo({ + sessionId: 'deadline-child', + sessionKind: 'subagent', + parentSessionId: parentSession.sessionId, + subagentEnabled: false, + availableSubagentSlots: [] + }) + const cancelConversation = vi.fn().mockResolvedValue(undefined) + const runtimePort = buildRuntimePort(parentSession, { + createSubagentSession: vi.fn().mockResolvedValue(childSession), + cancelConversation + }) + const tool = new SubagentOrchestratorTool(runtimePort as any) + + const definition = await tool.getToolDefinition(parentSession.sessionId) + const runTimeoutProperty = (definition?.function.parameters as any).properties.runTimeoutMs + expect(runTimeoutProperty).toMatchObject({ + type: 'number', + minimum: 1000, + maximum: 1800000 + }) + + await expect( + tool.call( + { + mode: 'parallel', + background: true, + runTimeoutMs: 999, + tasks: [{ slotId: 'reviewer', title: 'Invalid', prompt: 'Do not start.' }] + }, + parentSession.sessionId + ) + ).rejects.toThrow() + + const started = await tool.call( + { + mode: 'parallel', + background: true, + runTimeoutMs: 1200, + tasks: [{ slotId: 'reviewer', title: 'Deadline task', prompt: 'Keep running.' }] + }, + parentSession.sessionId + ) + const startedProgress = JSON.parse((started.rawData?.toolResult as any).subagentProgress) + + expect(startedProgress).toMatchObject({ + runTimeoutMs: 1200, + deadlineAt: Date.now() + 1200 + }) + expect(startedProgress.cancellationReason).toBeUndefined() + + await Promise.resolve() + await Promise.resolve() + await vi.advanceTimersByTimeAsync(1200) + + const info = await tool.call( + { operation: 'info', runId: startedProgress.runId }, + parentSession.sessionId + ) + const deadlineProgress = JSON.parse((info.rawData?.toolResult as any).subagentProgress) + + expect(deadlineProgress).toMatchObject({ + status: 'cancelled', + cancellationReason: 'Run deadline exceeded after 1200ms.', + runTimeoutMs: 1200, + deadlineAt: Date.now() + }) + expect(deadlineProgress.tasks[0]).toMatchObject({ + status: 'cancelled', + resultSummary: 'Run deadline exceeded after 1200ms.' + }) + expect(cancelConversation).toHaveBeenCalledWith(childSession.sessionId) + + const waited = await tool.call( + { operation: 'wait', runId: startedProgress.runId, timeoutMs: 0 }, + parentSession.sessionId + ) + expect((waited.rawData?.toolResult as any).subagentFinal).toBeTruthy() + } finally { + vi.useRealTimers() + } + }) + + it('waits for child cancellation but not tape discard after a deadline', async () => { + vi.useFakeTimers() + + try { + const parentSession = buildSessionInfo() + const childSession = buildSessionInfo({ + sessionId: 'slow-cancel-child', + sessionKind: 'subagent', + parentSessionId: parentSession.sessionId, + subagentEnabled: false, + availableSubagentSlots: [] + }) + let settleCancellation: (() => void) | undefined + const cancelConversation = vi.fn( + () => + new Promise((resolve) => { + settleCancellation = resolve + }) + ) + const discard = createDeferredPromise() + const discardSubagentTape = vi.fn(() => discard.promise) + const runtimePort = buildRuntimePort(parentSession, { + createSubagentSession: vi.fn().mockResolvedValue(childSession), + cancelConversation, + discardSubagentTape + }) + const tool = new SubagentOrchestratorTool(runtimePort as any) + + const started = await tool.call( + { + mode: 'parallel', + background: true, + runTimeoutMs: 1000, + tasks: [{ slotId: 'reviewer', title: 'Slow cancellation', prompt: 'Keep running.' }] + }, + parentSession.sessionId + ) + const runId = JSON.parse((started.rawData?.toolResult as any).subagentProgress).runId + await Promise.resolve() + await Promise.resolve() + + await vi.advanceTimersByTimeAsync(1000) + const info = await tool.call({ operation: 'info', runId }, parentSession.sessionId) + expect(JSON.parse((info.rawData?.toolResult as any).subagentProgress).status).toBe( + 'cancelled' + ) + expect(cancelConversation).toHaveBeenCalledWith(childSession.sessionId) + expect(discardSubagentTape).not.toHaveBeenCalled() + + settleCancellation?.() + await Promise.resolve() + await Promise.resolve() + const waited = await tool.call( + { operation: 'wait', runId, timeoutMs: 0 }, + parentSession.sessionId + ) + + expect(discardSubagentTape).toHaveBeenCalledWith( + parentSession.sessionId, + childSession.sessionId, + expect.objectContaining({ status: 'cancelled' }) + ) + expect((waited.rawData?.toolResult as any).subagentFinal).toBeTruthy() + + discard.resolve(undefined) + await Promise.resolve() + await Promise.resolve() + } finally { + vi.useRealTimers() + } + }) + + it('returns at the deadline when a completed child tape merge is still blocked', async () => { + vi.useFakeTimers() + + try { + const parentSession = buildSessionInfo() + const childSession = buildSessionInfo({ + sessionId: 'blocked-merge-child', + sessionKind: 'subagent', + parentSessionId: parentSession.sessionId, + subagentEnabled: false, + availableSubagentSlots: [] + }) + const merge = createDeferredPromise() + let listener: ((update: DeepChatInternalSessionUpdate) => void) | undefined + const runtimePort = buildRuntimePort(parentSession, { + createSubagentSession: vi.fn().mockResolvedValue(childSession), + subscribeDeepChatSessionUpdates: vi.fn((callback) => { + listener = callback + return () => { + listener = undefined + } + }), + mergeSubagentTape: vi.fn(() => merge.promise) + }) + const tool = new SubagentOrchestratorTool(runtimePort as any) + + const running = tool.call( + { + mode: 'chain', + runTimeoutMs: 1000, + tasks: [{ slotId: 'reviewer', title: 'Blocked merge', prompt: 'Finish normally.' }] + }, + parentSession.sessionId + ) + await vi.advanceTimersByTimeAsync(0) + listener?.({ + sessionId: childSession.sessionId, + kind: 'status', + updatedAt: Date.now(), + status: 'idle' + }) + await vi.advanceTimersByTimeAsync(0) + expect(runtimePort.mergeSubagentTape).toHaveBeenCalledOnce() + + await vi.advanceTimersByTimeAsync(1000) + + const result = await running + const finalProgress = JSON.parse((result.rawData?.toolResult as any).subagentFinal) + expect(finalProgress).toMatchObject({ + status: 'cancelled', + cancellationReason: 'Run deadline exceeded after 1000ms.' + }) + expect(finalProgress.tasks[0].status).toBe('completed') + expect(runtimePort.cancelConversation).not.toHaveBeenCalled() + + await expect( + tool.call({ operation: 'info', runId: finalProgress.runId }, parentSession.sessionId) + ).resolves.toBeTruthy() + await expect( + tool.call( + { operation: 'wait', runId: finalProgress.runId, timeoutMs: 0 }, + parentSession.sessionId + ) + ).resolves.toBeTruthy() + + merge.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + } finally { + vi.useRealTimers() + } + }) + + it('returns at the deadline when an errored child tape discard is still blocked', async () => { + vi.useFakeTimers() + + try { + const parentSession = buildSessionInfo() + const childSession = buildSessionInfo({ + sessionId: 'blocked-discard-child', + sessionKind: 'subagent', + parentSessionId: parentSession.sessionId, + subagentEnabled: false, + availableSubagentSlots: [] + }) + const discard = createDeferredPromise() + let listener: ((update: DeepChatInternalSessionUpdate) => void) | undefined + const runtimePort = buildRuntimePort(parentSession, { + createSubagentSession: vi.fn().mockResolvedValue(childSession), + subscribeDeepChatSessionUpdates: vi.fn((callback) => { + listener = callback + return () => { + listener = undefined + } + }), + discardSubagentTape: vi.fn(() => discard.promise) + }) + const tool = new SubagentOrchestratorTool(runtimePort as any) + + const running = tool.call( + { + mode: 'chain', + runTimeoutMs: 1000, + tasks: [{ slotId: 'reviewer', title: 'Blocked discard', prompt: 'Fail normally.' }] + }, + parentSession.sessionId + ) + await vi.advanceTimersByTimeAsync(0) + listener?.({ + sessionId: childSession.sessionId, + kind: 'status', + updatedAt: Date.now(), + status: 'error' + }) + await vi.advanceTimersByTimeAsync(0) + expect(runtimePort.discardSubagentTape).toHaveBeenCalledOnce() + + await vi.advanceTimersByTimeAsync(1000) + + const result = await running + const finalProgress = JSON.parse((result.rawData?.toolResult as any).subagentFinal) + expect(finalProgress).toMatchObject({ + status: 'cancelled', + cancellationReason: 'Run deadline exceeded after 1000ms.' + }) + expect(finalProgress.tasks[0].status).toBe('error') + expect(runtimePort.cancelConversation).not.toHaveBeenCalled() + + discard.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + } finally { + vi.useRealTimers() + } + }) + + it('registers cancellation before discarding a child created after the deadline', async () => { + vi.useFakeTimers() + + try { + const parentSession = buildSessionInfo() + const childSession = buildSessionInfo({ + sessionId: 'late-deadline-child', + sessionKind: 'subagent', + parentSessionId: parentSession.sessionId, + subagentEnabled: false, + availableSubagentSlots: [] + }) + const childCreation = createDeferredPromise() + const cancellation = createDeferredPromise() + const cancelConversation = vi.fn(() => cancellation.promise) + const discardSubagentTape = vi.fn().mockResolvedValue(undefined) + const runtimePort = buildRuntimePort(parentSession, { + createSubagentSession: vi.fn(() => childCreation.promise), + cancelConversation, + discardSubagentTape + }) + const tool = new SubagentOrchestratorTool(runtimePort as any) + + const started = await tool.call( + { + mode: 'parallel', + background: true, + runTimeoutMs: 1000, + tasks: [{ slotId: 'reviewer', title: 'Late child', prompt: 'Start after timeout.' }] + }, + parentSession.sessionId + ) + const runId = JSON.parse((started.rawData?.toolResult as any).subagentProgress).runId + + await vi.advanceTimersByTimeAsync(1000) + childCreation.resolve(childSession) + await vi.advanceTimersByTimeAsync(0) + + expect(cancelConversation).toHaveBeenCalledWith(childSession.sessionId) + await tool.call({ operation: 'info', runId }, parentSession.sessionId) + expect(discardSubagentTape).not.toHaveBeenCalled() + + cancellation.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + + expect(discardSubagentTape).toHaveBeenCalledWith( + parentSession.sessionId, + childSession.sessionId, + expect.objectContaining({ status: 'cancelled' }) + ) + } finally { + vi.useRealTimers() + } + }) + + it('limits each parent session to three active runs and frees capacity on cancellation', async () => { + vi.useFakeTimers() + + try { + const parentSession = buildSessionInfo() + let childIndex = 0 + const runtimePort = buildRuntimePort(parentSession, { + createSubagentSession: vi.fn().mockImplementation(async () => { + childIndex += 1 + return buildSessionInfo({ + sessionId: `active-child-${childIndex}`, + sessionKind: 'subagent', + parentSessionId: parentSession.sessionId, + subagentEnabled: false, + availableSubagentSlots: [] + }) + }) + }) + const tool = new SubagentOrchestratorTool(runtimePort as any) + const startRun = () => + tool.call( + { + mode: 'parallel', + background: true, + tasks: [{ slotId: 'reviewer', title: 'Active task', prompt: 'Keep running.' }] + }, + parentSession.sessionId + ) + + const attempts = await Promise.all( + Array.from({ length: 4 }, async () => { + try { + return { result: await startRun(), error: null } + } catch (error) { + return { result: null, error } + } + }) + ) + const startedRuns = attempts.filter((attempt) => attempt.result !== null) + const rejectedRuns = attempts.filter((attempt) => attempt.error !== null) + + expect(startedRuns).toHaveLength(3) + expect(rejectedRuns).toHaveLength(1) + expect(rejectedRuns[0]?.error).toEqual( + expect.objectContaining({ + message: 'A parent session can have at most 3 active subagent runs.' + }) + ) + + const activeRunIds = startedRuns.map( + ({ result }) => JSON.parse((result?.rawData?.toolResult as any).subagentProgress).runId + ) + + await tool.call({ operation: 'kill', runId: activeRunIds[0] }, parentSession.sessionId) + const replacement = await startRun() + activeRunIds.push(JSON.parse((replacement.rawData?.toolResult as any).subagentProgress).runId) + + for (const runId of activeRunIds.slice(1)) { + await tool.call({ operation: 'kill', runId }, parentSession.sessionId) + } + await Promise.resolve() + await Promise.resolve() + } finally { + vi.useRealTimers() + } + }) + it('starts background runs and supports list, info, and kill operations', async () => { const parentSession = buildSessionInfo() const childSession = buildSessionInfo({ @@ -561,77 +1033,186 @@ describe('SubagentOrchestratorTool', () => { warnSpy.mockRestore() }) - it('cancels a newly created child before handoff when the parent signal aborts', async () => { + it('rechecks cancellation after a blocked handoff and cancels again before tape discard', async () => { + vi.useFakeTimers() + + try { + const parentSession = buildSessionInfo() + const childSession = buildSessionInfo({ + sessionId: 'blocked-handoff-child', + sessionKind: 'subagent', + parentSessionId: parentSession.sessionId, + subagentEnabled: false, + availableSubagentSlots: [] + }) + const abortController = new AbortController() + const handoff = createDeferredPromise() + const cancellations = [createDeferredPromise(), createDeferredPromise()] + let cancellationIndex = 0 + const cancelConversation = vi.fn(() => cancellations[cancellationIndex++]!.promise) + const discardSubagentTape = vi.fn().mockResolvedValue(undefined) + const runtimePort = buildRuntimePort(parentSession, { + createSubagentSession: vi.fn().mockResolvedValue(childSession), + sendConversationMessage: vi.fn(() => handoff.promise), + cancelConversation, + discardSubagentTape + }) + const tool = new SubagentOrchestratorTool(runtimePort as any) + + const runPromise = tool.call( + { + mode: 'chain', + tasks: [ + { + slotId: 'reviewer', + title: 'Abort blocked handoff', + prompt: 'Do not wait for this handoff to settle.' + } + ] + }, + parentSession.sessionId, + { signal: abortController.signal } + ) + + await vi.advanceTimersByTimeAsync(0) + expect(runtimePort.sendConversationMessage).toHaveBeenCalledWith( + childSession.sessionId, + expect.any(String) + ) + + abortController.abort() + await expect(runPromise).rejects.toThrow('subagent_orchestrator cancelled.') + expect(cancelConversation).toHaveBeenCalledTimes(1) + expect(discardSubagentTape).not.toHaveBeenCalled() + + handoff.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + expect(cancelConversation).toHaveBeenCalledTimes(2) + + cancellations[0].resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + expect(discardSubagentTape).not.toHaveBeenCalled() + + cancellations[1].resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + expect(discardSubagentTape).toHaveBeenCalledWith( + parentSession.sessionId, + childSession.sessionId, + expect.objectContaining({ status: 'cancelled' }) + ) + } finally { + vi.useRealTimers() + } + }) + + it('observes foreground cancellation while child creation is blocked and cleans up later', async () => { + vi.useFakeTimers() + + try { + const parentSession = buildSessionInfo() + const childSession = buildSessionInfo({ + sessionId: 'late-created-child', + sessionKind: 'subagent', + parentSessionId: parentSession.sessionId, + subagentEnabled: false, + availableSubagentSlots: [] + }) + const abortController = new AbortController() + const childCreation = createDeferredPromise() + const cancellation = createDeferredPromise() + const cancelConversation = vi.fn(() => cancellation.promise) + const sendConversationMessage = vi.fn().mockResolvedValue(undefined) + const discardSubagentTape = vi.fn().mockResolvedValue(undefined) + const runtimePort = buildRuntimePort(parentSession, { + createSubagentSession: vi.fn(() => childCreation.promise), + sendConversationMessage, + cancelConversation, + discardSubagentTape + }) + const tool = new SubagentOrchestratorTool(runtimePort as any) + + const runPromise = tool.call( + { + mode: 'chain', + tasks: [ + { + slotId: 'reviewer', + title: 'Abort before creation', + prompt: 'Return cancellation before child creation finishes.' + } + ] + }, + parentSession.sessionId, + { signal: abortController.signal } + ) + + await vi.advanceTimersByTimeAsync(0) + expect(runtimePort.createSubagentSession).toHaveBeenCalledTimes(1) + + abortController.abort() + await expect(runPromise).rejects.toThrow('subagent_orchestrator cancelled.') + expect(cancelConversation).not.toHaveBeenCalled() + + childCreation.resolve(childSession) + await vi.advanceTimersByTimeAsync(0) + expect(sendConversationMessage).not.toHaveBeenCalled() + expect(cancelConversation).toHaveBeenCalledWith(childSession.sessionId) + expect(discardSubagentTape).not.toHaveBeenCalled() + + cancellation.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + expect(discardSubagentTape).toHaveBeenCalledWith( + parentSession.sessionId, + childSession.sessionId, + expect.objectContaining({ status: 'cancelled' }) + ) + } finally { + vi.useRealTimers() + } + }) + + it('returns cancellation while the parent workdir lookup remains blocked', async () => { const parentSession = buildSessionInfo() - const childSession = buildSessionInfo({ - sessionId: 'child-session', - agentName: 'Reviewer Clone', - sessionKind: 'subagent', - parentSessionId: parentSession.sessionId, - subagentEnabled: false, - availableSubagentSlots: [] - }) + const workdir = createDeferredPromise() const abortController = new AbortController() - let resolveCreate: ((value: ConversationSessionInfo) => void) | null = null - const createSubagentSession = vi.fn( - () => - new Promise((resolve) => { - resolveCreate = resolve - }) - ) - const sendConversationMessage = vi.fn().mockResolvedValue(undefined) - const cancelConversation = vi.fn().mockResolvedValue(undefined) - - const tool = new SubagentOrchestratorTool({ - resolveConversationWorkdir: vi.fn().mockResolvedValue(parentSession.projectDir), - resolveConversationSessionInfo: vi.fn().mockResolvedValue(parentSession), - createSubagentSession, - sendConversationMessage, - cancelConversation, - subscribeDeepChatSessionUpdates: vi.fn(() => () => undefined), - getSkillPresenter: vi.fn(() => ({})), - getYoBrowserToolHandler: vi.fn(() => ({})), - getFilePresenter: vi.fn(() => ({ - getMimeType: vi.fn(), - prepareFileCompletely: vi.fn() - })), - getLlmProviderPresenter: vi.fn(() => ({ - executeWithRateLimit: vi.fn().mockResolvedValue(undefined), - generateCompletionStandalone: vi.fn(), - generateImageStandalone: vi.fn() - })), - createSettingsWindow: vi.fn(), - sendToWindow: vi.fn(), - getApprovedFilePaths: vi.fn(() => []), - consumeSettingsApproval: vi.fn(() => false) - } as any) + const runtimePort = buildRuntimePort(parentSession, { + resolveConversationWorkdir: vi.fn(() => workdir.promise) + }) + const tool = new SubagentOrchestratorTool(runtimePort as any) - const runPromise = tool.call( + const running = tool.call( { mode: 'chain', tasks: [ { slotId: 'reviewer', - title: 'Abort before handoff', - prompt: 'Cancel this before the handoff is sent.' + title: 'Blocked parent lookup', + prompt: 'Do not start after cancellation.' } ] }, parentSession.sessionId, - { - signal: abortController.signal - } + { signal: abortController.signal } ) - - for (let index = 0; index < 10 && createSubagentSession.mock.calls.length === 0; index += 1) { - await new Promise((resolve) => setTimeout(resolve, 0)) - } + await vi.waitFor(() => expect(runtimePort.resolveConversationWorkdir).toHaveBeenCalledOnce()) abortController.abort() - resolveCreate?.(childSession) - await expect(runPromise).rejects.toThrow('subagent_orchestrator cancelled.') - expect(sendConversationMessage).not.toHaveBeenCalled() - expect(cancelConversation).toHaveBeenCalledWith(childSession.sessionId) + await expect(running).rejects.toThrow('subagent_orchestrator cancelled.') + expect(runtimePort.createSubagentSession).not.toHaveBeenCalled() + workdir.resolve(parentSession.projectDir) + }) + + it('clears a wait timeout when run completion wins the race', async () => { + vi.useFakeTimers() + + try { + const tool = new SubagentOrchestratorTool({} as any) + await (tool as any).waitForRunCompletion({ completion: Promise.resolve() }, 300000) + + expect(vi.getTimerCount()).toBe(0) + } finally { + vi.useRealTimers() + } }) }) diff --git a/test/main/presenter/toolPresenter/toolPresenter.test.ts b/test/main/presenter/toolPresenter/toolPresenter.test.ts index a13690245..23eb992f5 100644 --- a/test/main/presenter/toolPresenter/toolPresenter.test.ts +++ b/test/main/presenter/toolPresenter/toolPresenter.test.ts @@ -656,6 +656,106 @@ describe('ToolPresenter', () => { ) }) + it('forwards cancellation and stored access context to MCP permission pre-checks', async () => { + const mcpPresenter = { + getAllToolDefinitions: vi + .fn() + .mockResolvedValue([buildToolDefinition('mcp_only', 'server-a')]), + callTool: vi.fn(), + preCheckToolPermission: vi.fn().mockResolvedValue(null) + } as any + const toolPresenter = new ToolPresenter({ + mcpPresenter, + configPresenter: { + getSkillsEnabled: vi.fn().mockReturnValue(false), + getSkillsPath: vi.fn().mockReturnValue('C:\\skills'), + getModelConfig: vi.fn() + } as any, + commandPermissionHandler: new CommandPermissionService(), + agentToolRuntime: buildAgentToolRuntimeMock() + }) + const abortController = new AbortController() + await toolPresenter.getAllToolDefinitions({ + agentId: 'agent-1', + enabledMcpServerIds: ['server-a'], + chatMode: 'agent', + conversationId: 'session-1' + }) + const request = { + id: 'permission-1', + type: 'function' as const, + function: { name: 'mcp_only', arguments: '{}' }, + conversationId: 'session-1' + } + + await toolPresenter.preCheckToolPermission(request, { + permissionMode: 'default', + signal: abortController.signal + }) + + expect(mcpPresenter.preCheckToolPermission).toHaveBeenCalledWith(request, { + agentId: 'agent-1', + enabledServerIds: ['server-a'], + signal: abortController.signal + }) + }) + + it('observes a late agent permission failure after pre-check synchronously cancels', async () => { + let rejectPermission!: (reason?: unknown) => void + const permission = new Promise((_, reject) => { + rejectPermission = reject + }) + const mcpPresenter = { + getAllToolDefinitions: vi.fn().mockResolvedValue([]), + callTool: vi.fn() + } as any + const toolPresenter = new ToolPresenter({ + mcpPresenter, + configPresenter: { + getSkillsEnabled: vi.fn().mockReturnValue(false), + getSkillsPath: vi.fn().mockReturnValue('C:\\skills'), + getModelConfig: vi.fn() + } as any, + commandPermissionHandler: new CommandPermissionService(), + agentToolRuntime: buildAgentToolRuntimeMock() + }) + const abortController = new AbortController() + const lateError = new Error('late permission failure') + const unhandled = vi.fn() + await toolPresenter.getAllToolDefinitions({ + chatMode: 'agent', + conversationId: 'permission-cancel-session' + }) + const agentToolManager = (toolPresenter as any).agentToolManager + agentToolManager.preCheckToolPermission = vi.fn().mockImplementation(() => { + abortController.abort() + return permission + }) + + await expect( + toolPresenter.preCheckToolPermission( + { + id: 'permission-sync-cancel', + type: 'function', + function: { name: UPDATE_PLAN_TOOL_NAME, arguments: '{}' }, + conversationId: 'permission-cancel-session' + }, + { signal: abortController.signal } + ) + ).rejects.toMatchObject({ name: 'AbortError' }) + expect(agentToolManager.preCheckToolPermission).toHaveBeenCalledTimes(1) + + process.on('unhandledRejection', unhandled) + try { + rejectPermission(lateError) + await new Promise((resolve) => setImmediate(resolve)) + await new Promise((resolve) => setImmediate(resolve)) + expect(unhandled.mock.calls.some(([reason]) => reason === lateError)).toBe(false) + } finally { + process.off('unhandledRejection', unhandled) + } + }) + it('preserves unrestricted MCP policy in stored conversation context', async () => { const mcpPresenter = { getAllToolDefinitions: vi @@ -675,6 +775,7 @@ describe('ToolPresenter', () => { commandPermissionHandler: new CommandPermissionService(), agentToolRuntime: buildAgentToolRuntimeMock() }) + const abortController = new AbortController() await toolPresenter.getAllToolDefinitions({ agentId: 'agent-1', @@ -683,24 +784,28 @@ describe('ToolPresenter', () => { conversationId: 'session-unrestricted' }) - await toolPresenter.callTool({ - id: 'tool-1', - type: 'function', - function: { - name: 'mcp_only', - arguments: '{}' - }, - server: { - name: 'open-server' - }, - conversationId: 'session-unrestricted' - } as any) + await toolPresenter.callTool( + { + id: 'tool-1', + type: 'function', + function: { + name: 'mcp_only', + arguments: '{}' + }, + server: { + name: 'open-server' + }, + conversationId: 'session-unrestricted' + } as any, + { signal: abortController.signal } + ) expect(mcpPresenter.callTool).toHaveBeenCalledWith( expect.objectContaining({ conversationId: 'session-unrestricted' }), expect.objectContaining({ agentId: 'agent-1', - enabledServerIds: undefined + enabledServerIds: undefined, + signal: abortController.signal }) ) }) diff --git a/test/main/scripts/architectureGuard.test.ts b/test/main/scripts/architectureGuard.test.ts index 79542ada7..7d22d05d6 100644 --- a/test/main/scripts/architectureGuard.test.ts +++ b/test/main/scripts/architectureGuard.test.ts @@ -523,7 +523,7 @@ describe('architecture guard', () => { beforeAll(async () => { violations = await runArchitectureGuard({ virtualFiles }) - }) + }, 30_000) it('passes against the current production source through the CLI', () => { const result = spawnSync(process.execPath, ['scripts/architecture-guard.mjs'], {