Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions docs/architecture/agent-memory-system/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ flowchart TD
| Runtime wiring | `agentRuntimePresenter/index.ts` | coordinator construction/dependencies and thin instance/lifecycle delegates; no duplicate orchestration maps |
| Runtime | `agentRuntimePresenter/tapeService.ts` | `search()` / `getContext()` / `ensureSearchProjection()` |
| Tools | `toolPresenter/agentTools/agentMemoryTools.ts` | `memory_remember` / `memory_recall` / `memory_forget` |
| Tools | `toolPresenter/agentTools/agentTapeTools.ts` | `tape_info` / `tape_search` / `tape_context` / `tape_anchors` / `tape_handoff` |
| Tools | `toolPresenter/agentTools/agentTapeTools.ts` | Atomic model recall pair: `tape_search` / `tape_context`; info, anchors, and handoff remain below the model-tool boundary |
| Skills | `resources/skills/memory-management/SKILL.md` | Discoverable guidance for recall/remember discipline and Memory vs Skill vs Scheduled Task routing |
| Contracts | `shared/contracts/routes/memory.routes.ts` | All `memory.*` IPC routes + DTO schemas |
| Contracts | `shared/contracts/events/memory.events.ts` | `memory.updated` event plus the schema-derived authoritative update-reason type |
Expand Down Expand Up @@ -717,9 +717,11 @@ flowchart TD
BM25-ranked with a `LIKE` fallback/supplement.
- **`tape_context`** expands specific entry ids with a bounded window (before/after default 2, clamp ≤ 20),
an entry cap (default 50), and byte budgets (per-entry default 2048 / max 8192; total default 16384 / max
65536), UTF-8-safe truncated. Tape tools are DeepChat-agent-only and `tape_context` is advertised only when
the tool-runtime port exposes `getTapeContext` (wired directly to `SessionProjectionCoordinator`, not the
memory kernel; in practice always present).
65536), UTF-8-safe truncated. The recall pair is DeepChat-agent-only and is advertised atomically only when
both `searchTape` and `getTapeContext` runtime ports are present for a persisted DeepChat session. These
ports are wired directly to `SessionProjectionCoordinator`, not the memory kernel.
- Memory extraction and injection do not invoke either model tool. They consume the effective Tape and exact
lineage through internal runtime contracts, so model-facing Tape exposure does not gate Memory behavior.
- The whole projection/search layer is fail-open: any error degrades to a coarser search over the effective
tape rather than throwing (the one exception is an unparseable time boundary, which is reported).

Expand Down Expand Up @@ -822,9 +824,12 @@ This is where the "stabilization" and "kernel hardening" work concentrates.
| `agent-memory` | `memory_remember` | Persist a durable fact/event with optional category; routes through the decision ring (`coordinateWrite`). |
| `agent-memory` | `memory_recall` | Recall relevant memories for a query (ranking/limit are kernel-side). |
| `agent-memory` | `memory_forget` | **Archive** (soft delete) a memory by id so it is no longer recalled. |
| `agent-tape` | `tape_info` / `tape_anchors` / `tape_handoff` | Tape introspection and subagent handoff. |
| `agent-tape` | `tape_search` | Overview-only search of the tape projection (no payload). |
| `agent-tape` | `tape_context` | Bounded evidence expansion of specific entry ids. |
| `agent-tape` | `tape_search` | Overview-only search of the tape projection (no payload); exposed only as part of the recall pair. |
| `agent-tape` | `tape_context` | Bounded evidence expansion of specific entry ids; exposed only as part of the recall pair. |

`tape_info` and `tape_anchors` remain lower-level diagnostic APIs. `tape_handoff` remains a runtime-only
transition API and requires a non-empty durable summary. All three names stay reserved but are absent from
model definitions, prompts, configuration UI, and Agent tool dispatch.

Tool results are success-enveloped even for "soft" outcomes (memory disabled, `noop`, not-found) — callers
inspect `result.ok`, not `isError`. Hard infra failures throw.
Expand Down
3 changes: 2 additions & 1 deletion docs/architecture/deepchat-tape-baseline/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ DeepChat already has the main Tape primitives.
| View policy | `tapeViewPolicy.ts` | Registry and selector boundary; `legacy_context_v1` delegates to the current selector. |
| Context selection | `contextBuilder.ts` | Legacy token-budget selector used by `legacy_context_v1`. |
| Request trace | `deepchat_message_traces` | Stores redacted provider request previews for the trace dialog. |
| Agent tools | `agentTapeTools.ts` | Exposes `tape_info`, `tape_search`, `tape_anchors`, `tape_handoff`. |
| Model recall tools | `agentTapeTools.ts` | Exposes the atomic `tape_search` / `tape_context` pair only when both runtime ports are available for a persisted DeepChat session. |
| Diagnostic/runtime APIs | `DeepChatTapeService` + `SessionProjectionCoordinator` | Retains info, anchor listing, and handoff below the model-tool boundary; these capabilities are not user-configurable Agent tools. |

The first implementation step uses this baseline as the single runtime path. `DeepChatTapeService`
remains the Tape service boundary.
Expand Down
13 changes: 10 additions & 3 deletions docs/architecture/session-boundary-cleanup/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,14 +152,21 @@ AgentSessionPresenter
### Startup tasks

- Existing task IDs, targets, phases, resource classes, label keys, priorities, and non-critical
failure behavior remain unchanged.
failure behavior remain unchanged unless a later migration explicitly supersedes that task. The
disabled-tool cleanup is superseded by `main:disabled-agent-tool-cleanup` with label key
`startup.main.disabledAgentToolCleanup`; its target, phase, resource class, priority, and
non-critical failure behavior remain unchanged.
- Legacy import remains single-flight and preserves persisted status and retry behavior.
- Usage backfill remains single-flight, stale-running detection remains unchanged, and dashboard
reads continue to expose its normalized status.
- Mainline normalization retains `sqlite-mainline-normalization-v1`, batch size, cursor ordering,
progress writes, yielding, completion metadata, and failure metadata.
- Disabled-tool cleanup retains `agent-disabled-search-tool-cleanup-v1`, session and agent-config
cleanup behavior, progress yielding, and completion/failure metadata.
- Disabled-tool cleanup preserves the historical `agent-disabled-search-tool-cleanup-v1` record but
uses the independent `agent-disabled-tool-capability-cleanup-v2` completion gate so users who
completed v1 still receive Tape capability cleanup. Session and agent-config cleanup remains
idempotent, bounded, yielding, and retryable with completion/failure metadata. Session cleanup
updates the legacy JSON and normalized disabled-tool rows transactionally without changing
`updated_at` or environment recency.

### History search

Expand Down
101 changes: 101 additions & 0 deletions docs/architecture/tape-capability-layering/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Tape Capability Layering - Plan

## Capability Policy

- Define `AgentToolExposure` as `user-configurable`, `system-model`, `runtime-only`, or
`diagnostic` in a shared Agent tool policy module.
- Keep the five Tape names and their classifications in one registry. Treat unregistered existing
Agent tools as `user-configurable` to avoid unrelated behavior changes.
- Require every Agent tool assembly site to state its expected exposure and assert that it matches
the shared registry. This preserves the compatibility fallback while preventing a capability
registered as internal from silently entering the configurable catalog.
- Keep exposure metadata out of provider-bound `MCPToolDefinition` values. Main-process catalog
selection and execution checks consume the policy directly.

## Catalog Separation

- Keep `ToolPresenter.getAllToolDefinitions()` as the runtime catalog. Apply
`disabledAgentTools` only to `user-configurable` definitions and publish only this catalog into
the per-conversation `ToolMapper`.
- Track the conversation provenance of the compatibility mapper. A published per-conversation
mapper is authoritative; an absent tool, or a missing/cleared mapper while the compatibility
snapshot belongs to another conversation, is unavailable. Preserve only the no-ID draft snapshot
as the bridge into the first persisted turn and for legacy no-ID calls.
- Add `getConfigurableAgentToolDefinitions()` for renderer configuration. Return only
`user-configurable` Agent definitions and do not resolve MCP definitions, publish mappings, or
remember conversation MCP access.
- Preserve the existing IPC route name and payload while routing it to the configurable catalog.
Rename the renderer client method so callers cannot confuse it with the runtime catalog.
- Update the new-thread tool popover and Agent settings to consume the configurable catalog without
hard-coded Tape-name filtering.

### Renderer Behavior

```text
BEFORE

New-thread draft -> runtime catalog -> no agent-tape group (no persisted session)
Persisted DeepChat thread -> runtime catalog -> agent-tape group with user-facing switches

AFTER

New-thread draft -> configurable catalog -> user-configurable Agent tools only
Persisted DeepChat thread -> configurable catalog -> user-configurable Agent tools only
```

## Tape Model Boundary

- Restrict `AgentTapeToolHandler` definitions and execution to the atomic `tape_search` and
`tape_context` recall pair.
- Require a DeepChat session plus both recall runtime ports before advertising either definition.
- Remove diagnostic and handoff descriptions from the tool system prompt.
- Narrow `AgentToolRuntimePort` to model-callable Tape operations. Retain Tape info, anchor listing,
and handoff on the lower runtime/application Tape boundary for future diagnostics and runtime use.
- Reject known non-model Tape names at the Agent tool execution boundary, including stale deferred
calls, while keeping their historical facts replayable.
- Revalidate the DeepChat session and complete recall pair at execution time before calling either
runtime port.

## Handoff and Persistence

- Require a trimmed non-empty summary for runtime handoff. Validate before appending and continue to
derive cursor, range, and source message IDs from the effective Tape.
- Leave automatic compaction and context-overflow recovery on their existing compare-and-set path.
- Normalize all disabled-tool writes to drop non-configurable Tape names.
- Add a separately keyed, idempotent startup cleanup for existing session and Agent configuration
rows. Reuse the existing bounded-yield migration pattern and do not alter transcript or Tape data.
- Persist session cleanup through a migration-owned transaction that updates both disabled-tool
stores without touching `updated_at` or synchronizing environment activity.

## Compatibility

- Keep the IPC route wire shape, provider tool shape, Tape schema, replay schema, and database schema
unchanged.
- Keep Tape names reserved so an MCP server cannot reinterpret a historical/internal name.
- A downgraded build may show Tape tools again after the cleanup removed obsolete disabled values;
no conversation or Tape data is lost, and the upgraded build remains deterministic.
- Leave pre-existing untracked documentation untouched. Update only maintained tracked contracts
that describe the old five-tool model surface.

## Test Strategy

- Cover runtime/configurable catalog membership and prove configurable reads have no mapper/access
side effects.
- Cover concurrent conversation mappings and prove a missing, cleared, or incomplete conversation
catalog cannot resolve through another conversation's latest mapping, while draft-origin mapping
compatibility remains intact.
- Cover DeepChat, ACP, missing-session, missing-port, stale-disabled-list, same-name MCP, and deferred
call behavior.
- Cover pre-append handoff validation and derived provenance for valid handoffs.
- Cover v2 cleanup idempotence, preservation of ordinary disabled tools, Agent config cleanup, and
bounded session iteration. Use a real SQLite test to preserve session ordering and environment
recency.
- Run focused main and renderer tests before each implementation commit, then the full repository
validation sequence before final handoff.

## Commit Slices

1. `docs(tape): define capability layering`
2. `refactor(tools): split tool catalogs`
3. `fix(tape): limit model capabilities`
4. `fix(config): remove tape tool toggles`
92 changes: 92 additions & 0 deletions docs/architecture/tape-capability-layering/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Tape Capability Layering - Spec

> Status: **implemented**

## Problem

DeepChat currently uses the runtime tool catalog as the source for both model execution and the
renderer's Agent tool configuration. Tape tools require a persisted DeepChat session, so the
configuration UI changes after the first message creates a session. The same catalog also applies
`disabledAgentTools` to every built-in Agent tool, which incorrectly presents Tape runtime
capabilities as optional user tools.

This behavior is reported by
[GitHub issue #1975](https://github.com/ThinkInAIXYZ/deepchat/issues/1975).

## Requirements

1. Tape recording, reconstruction, compaction, replay, and memory lineage remain always-on runtime
infrastructure with no user-facing disable switch.
2. Agent tools have one authoritative exposure classification:

| Tool | Exposure |
| --- | --- |
| `tape_search` | `system-model` |
| `tape_context` | `system-model` |
| `tape_info` | `diagnostic` |
| `tape_anchors` | `diagnostic` |
| `tape_handoff` | `runtime-only` |

3. Existing Agent tools without an explicit classification remain `user-configurable`.
4. The runtime catalog includes available `system-model` tools regardless of
`disabledAgentTools`. The configurable catalog contains only `user-configurable` Agent tools.
5. Configurable catalog reads must not publish a `ToolMapper`, update conversation MCP access
context, or mutate runtime tool-profile caches.
6. The model-facing Tape contract exposes `tape_search` and `tape_context` as an atomic recall pair.
Neither tool is exposed if either runtime port is unavailable.
7. `tape_info`, `tape_anchors`, and `tape_handoff` remain reserved names but cannot be invoked as
model tools.
8. Runtime handoff requires a non-empty durable summary before advancing the reconstruction cursor.
9. Persisted Tape names are removed from Agent and session disabled-tool lists without modifying
messages, Tape entries, or replay manifests.
10. A published per-conversation runtime catalog is authoritative for execution. A missing tool in
that catalog, or a missing/cleared mapper while the global snapshot belongs to another
conversation, cannot fall back across conversations. A global snapshot published without a
conversation ID remains available as the draft-to-persisted-session compatibility bridge.
11. Tape recall execution revalidates the persisted DeepChat session and complete runtime pair so
stale, direct, or fabricated calls cannot bypass catalog availability.
12. Capability cleanup changes only disabled-tool state; it preserves session activity timestamps,
conversation ordering, and environment recency.
13. Every Agent tool assembly site declares its expected exposure and verifies it against the
shared exposure policy before adding definitions to a catalog.

## Acceptance Criteria

- No `agent-tape` group appears in either a new-thread draft or a persisted session. Other
configurable Agent tool groups may continue to vary with session-scoped capabilities such as
Memory, image generation, subagents, and active Skills.
- A DeepChat runtime session exposes exactly `tape_search` and `tape_context` from the Tape tool
group, even when a legacy disabled list contains those names.
- ACP sessions, missing conversation IDs, and incomplete recall ports expose no Tape model tools.
- Direct or deferred model calls to diagnostic/runtime-only Tape names fail without appending an
anchor or otherwise mutating Tape state.
- Empty or whitespace-only runtime handoff summaries fail before an anchor is appended.
- The disabled-tool cleanup is idempotent, preserves ordinary disabled tools, and yields while
processing large session sets without changing session or environment recency.
- A request carrying a conversation ID cannot execute through another conversation's latest
mapping when its own catalog is missing, cleared, or does not contain the tool, while a draft
catalog published without an ID remains usable for the first persisted turn.
- Memory enablement, memory extraction/injection, subagents, skills, MCP tools, and ordinary Agent
tool switches retain their existing behavior.
- Focused tests, typecheck, full tests, formatting, i18n validation, and lint pass.

## Constraints

- Keep the `tools.listDefinitions` IPC route name and wire shape compatible.
- Keep all Tape names reserved against same-name MCP definitions.
- Preserve historical messages, tool-call facts, Tape entries, and replay behavior.
- Do not modify or stage pre-existing untracked documentation in the working tree.
- Do not push, create a pull request, tag, or mutate the linked GitHub issue.

## Non-goals

- Implementing a phase-aware or topic-aware Tape view.
- Implementing an anchor graph, cross-anchor traversal, or a second `TapeViewPolicy`.
- Adding a Tape inspector or special-purpose Tape trace UI.
- Reclassifying Memory, Subagent, Skill, Browser, or other non-Tape Agent tools.
- Making every configurable Agent tool group identical between draft and persisted sessions.
- Making Bub's current tool exposure policy a compatibility target.

## Open Questions

None.
14 changes: 14 additions & 0 deletions docs/architecture/tape-capability-layering/tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Tape Capability Layering - Tasks

- [x] T1: Record the capability matrix, catalog contracts, compatibility rules, and non-goals.
- [x] T2: Add the exposure policy and separate runtime from configurable Agent tool catalogs.
- [x] T3: Move renderer configuration callers to the side-effect-free configurable catalog.
- [x] T4: Restrict the model-facing Tape surface to the search/context recall pair.
- [x] T5: Make runtime handoff reject empty inherited summaries before appending an anchor.
- [x] T6: Remove non-configurable Tape names from new and persisted disabled-tool state.
- [x] T7: Update maintained tracked architecture contracts and focused regression tests.
- [x] T8: Run format, i18n validation, lint, typecheck, and the full test suite.
- [x] T9: Make per-conversation mappings authoritative and revalidate Tape recall on execution.
- [x] T10: Preserve session and environment recency during disabled-tool capability cleanup.
- [x] T11: Require explicit exposure expectations at Agent tool assembly sites.
- [x] T12: Align acceptance criteria and add regression coverage for the hardened boundaries.
Loading
Loading