fix(hermes): route memory and context safely

[ScriptedAlchemy]

Summary

This PR turns the Hermes integration into a session-safe TraceDecay client, adds a real profile-level memory/session scope for conversations that do not belong to a repository, connects Hermes to its native progressive tool and skill discovery mechanisms, and hardens TraceDecay's autonomous self-improvement machinery so it can safely learn from completed turns across Hermes, Codex, Cursor, Claude, Kiro, and the other supported transcript providers.

The architectural rule is:

TraceDecay self-improvement is Hermes-like, not Hermes-triggered.

Hermes supplies one stream of session evidence. TraceDecay owns storage scope, ingestion, reflection, curation, validation, scheduling, locking, managed-skill transactions, and native host exports. Telegram is only one possible Hermes transport; correlation is keyed by Hermes session and workspace evidence, so CLI, Desktop, gateway, Telegram, and future transports follow the same rules.

This description is intentionally detailed because the final implementation spans storage architecture, project identity, live and historical transcript routing, memory mutation, LCM routing, autonomous curation, managed-skill lifecycle safety, and host integration behavior.

Storage model: project scope and user scope are separate databases

TraceDecay still has one user-owned profile root, normally ~/.tracedecay/, but it does not store every fact in one global table with a project_id column.

The final layout is:

~/.tracedecay/global.db
└── project registry, aliases, persistent identities, store locations, accounting

~/.tracedecay/user-memory.db
└── durable user preferences and facts learned from projectless conversations

~/.tracedecay/user-sessions.db
└── projectless messages, transcript projections, raw LCM material, summaries

~/.tracedecay/user-automation/
└── profile-level automation artifacts, ledgers, proposals, locks, outcomes

~/.tracedecay/projects/<project_id>/tracedecay.db
└── project facts, entities, feedback, vectors, holographic banks

~/.tracedecay/projects/<project_id>/sessions.db
└── project messages, transcript projections, raw LCM material, summaries

This means routing must be correct before a fact/session database is opened. A bad cwd does not merely attach an incorrect tag: it can open the wrong shard. Every host path now resolves through TraceDecay's registered project identity before project storage is selected. If that identity cannot be proven, the operation either routes to user scope when projectless behavior is valid or fails closed.

Hermes project routing

Hermes project selection now follows this order:

1. An explicit project root supplied for the active Hermes session.
2. Stock Hermes's authoritative logical workspace from agent.runtime_cwd.
3. TERMINAL_CWD for compatible terminal integrations.
4. Process cwd only as a final compatibility candidate.
5. TraceDecay resolves the candidate to a registered project/Git identity.
6. The resolved root must contain the candidate or otherwise match the registered identity.
7. Project facts and project LCM then open only that project's shard.

An uninitialized directory cannot inherit whichever project happens to be active in another gateway session. A descendant such as <project>/src/nested correctly resolves to <project>, while a sibling or unrelated gateway directory does not.

The context engine now stores project_root inside its per-session state. Independent sessions can be project-bound or untethered at the same time. Returning to an existing session restores its prior route instead of overwriting it with host config or another session's cwd. Compression/session rotation inherits the previous session state where Hermes provides an old session identifier. Stock Hermes deep-copy behavior creates a fresh lock, preserves routing and token-budget state, and drops live agent references.

Hermes session correlation, not Telegram correlation

Live correlation is keyed by the Hermes session ID and completed turn. It is deliberately independent of Telegram.

For each successful completed turn, the plugin:

• projects only the completed user/assistant turn rather than replaying a whole mixed-session transcript;
• preserves or synthesizes stable message IDs and timestamps;
• records Hermes provider/session provenance;
• uses the session's authoritative workspace when it resolves to a registered project;
• examines structured tool arguments from that exact turn when a session-level cwd is absent;
• recognizes project roots/paths/selectors, cwd, workdir, and terminal-cwd evidence;
• gives exact turn-level tool routing precedence over stale session-level cwd;
• validates any inferred project before writing;
• routes an unproven turn to profile-level user LCM rather than another project;
• emits turnCompleted only after the foreground turn completes;
• emits turnIngested only after LCM ingestion succeeds;
• schedules background review from the successful ingestion receipt without delaying the response.

lcm_preflight(transcript_projection=true) upserts the stable completed-turn projection with host live-route provenance. This makes Hermes turns searchable through normal session/message search while preserving the LCM engine's compaction semantics.

The post-tool hook also emits bounded asynchronous terminal receipts. Receipt-triggered reviews are serialized at the profile level and only run after successful ingestion, preventing foreground failures from producing false evidence.

Historical Hermes correlation

Historical import uses the same storage rules. Strong persisted evidence includes a supported profile pin, persisted session cwd, registered Git root, or structured project selectors in recorded tool calls.

For old default-profile sessions that have no trustworthy cwd/root, import groups records into turns and assigns only the turns whose own structured tool arguments prove a project. It does not assign an entire long-running chat to one repository merely because one tool call touched that repository. Turn-level explicit evidence wins over a stale session cwd, preventing the same turn from appearing in multiple project shards.

The historical cursor/version was advanced so existing users receive a bounded safe rescan. Backfill coalesces provider-wide historical work into one review signal instead of launching one review per discovered file/session.

Profile-level user memory

The PR adds first-class memory_scope=project|user handling across the CLI/MCP handlers and Hermes provider bridge.

• Project decisions, code knowledge, repository conventions, and implementation facts use the active project shard.
• Durable preferences and facts meant to follow the person across repositories use user scope.
• Untethered/general chat uses user scope.
• User facts can be added, searched, probed, listed, updated, merged, and removed with the same validation model as project facts.
• A user-scoped operation cannot also carry a project selector.
• Explicit cross-project selectors remain read-only; cross-project mutation is rejected.
• Project retrieval merges active-project and user facts, deduplicates results, and preserves scope/provenance.
• User facts are not copied into every project shard.

Hermes's memory provider now routes target="user", user-preference categories, and projectless writes to user-memory.db. Project facts remain bound to the provider instance's current Hermes session project.

The model-visible memory surface remains compact: fact_store, fact_feedback, and memory_status. Provider-owned tools are not registered a second time under prefixed names when TraceDecay is already the active memory provider.

Profile-level user sessions and LCM

Projectless conversations now have a real session store rather than being dropped or forced into a gateway/project directory.

The LCM APIs accept storage_scope=project|user. User scope opens user-sessions.db, rejects project selectors, and supports the same status, replay, grep, describe, expand, preflight, compress, boundary, doctor, and transcript-projection machinery where applicable.

Hermes derives this scope internally from its session state. storage_scope remains available in the raw TraceDecay CLI/MCP schemas, but the Hermes plugin strips it (and legacy hermes_home) from agent-visible schemas so an agent cannot override the session's authoritative route. The plugin injects storage_scope=user only when there is no verified project.

This also fixes the earlier behavior where projectless Hermes turns skipped LCM entirely. They now remain searchable and compressible in the profile-level session store.

Codex, Cursor, Claude, Kiro, and other projectless transcript providers

The project/user distinction is not Hermes-only.

The ingestion adapters now route unattributed sessions from:

• Hermes;
• Codex live events and history;
• Cursor JSONL live/history and composer-only history;
• Claude live/history;
• Kiro;
• Cline, Roo, and Kilo-compatible transcripts;
• Vibe;

into user-sessions.db, while excluding sessions/turns that can be proven to belong to a registered project. The registry check fails closed: if project registration cannot be inspected safely, user ingestion does not risk copying project material into profile-level chat history.

Provider-specific edge cases fixed during final review include:

• Codex sessions that switch cwd mid-session no longer leak later registered-project turns into user storage.
• Cursor hook events without cwd do not use a transcript storage slug as project evidence.
• Cursor composer-owned sessions deduplicate the overlapping JSONL sweep.
• Claude user sessions keep neutral user-scope provenance rather than presenting the profile directory as a code project.
• Hermes registered session cwd fallback cannot be reclassified as projectless user material.
• Kiro and the remaining adapters use bounded import and the same registered-root exclusion.

Kimi, OpenCode, Gemini, Copilot, and Zed do not currently expose equivalent local transcript parsers/hooks in TraceDecay. Their generated guidance explicitly directs projectless durable preferences to memory_scope=user; this PR does not claim ingestion surfaces those hosts do not provide.

Autonomous self-improvement semantics

Autonomous TraceDecay curation no longer stops at a misleading dry-run gate.

The final policy is:

• Scheduler/background/session-receipt automation validates proposals and applies accepted memory mutations.
• The legacy auto_apply_memory_ops setting is retained for configuration compatibility but is not an autonomous no-op gate.
• Failed or partial application reports apply_incomplete, not dry_run_only.
• Internal reports say validated_before_apply, describing the actual sequence.
• Explicit human/operator diagnostic preview APIs remain read-only.
• Destructive offline migrations, database repair/GC, and code-edit tools retain explicit preview/confirmation modes because those operations are not autonomous memory curation.

The last point is deliberate. There should not be a dry-run default in the autonomous learning loop, but removing every preview implementation from the system would weaken operator safety for migrations, repair, GC, and source edits. The dashboard memory-curation preview was also corrected so apply=false performs no derived-vector or holographic-bank repair; a preview is now genuinely read-only.

Memory curation supports validated add, update, merge, delete, deterministic conflict handling, entity/tag/trust maintenance, missing-vector repair, and dirty holographic-bank rebuilding. Agents can remove or supersede memories that are no longer relevant, but only through the same bounded validation, evidence, trust, and reversible audit machinery used for other mutations.

The new memory_fact_relations table is also covered by profile-shard consolidation. Relation endpoints are remapped through the consolidated fact-ID map, newer relation metadata wins deterministically, self-relations created by fact deduplication are skipped, and interrupted/retried consolidation preserves one relation row without duplication.

User-scope automation

Projectless evidence has a complete automation home under ~/.tracedecay/user-automation/.

It has independent artifacts, run ledger, proposals, outcomes, and locks. Session reflection reads user-sessions.db and writes accepted facts into user-memory.db. User memory curation operates on the user fact/vector/bank store. Skill writing can learn provider-independent workflow patterns from bounded user-session evidence.

When the global config has no explicit [automation] table, user automation receives the autonomous defaults: enabled Codex app-server backend, session reflector/memory curator/skill writer enabled, and validated memory/skill activation enabled. An explicitly configured global table or user-automation sidecar remains authoritative, including explicit disablement.

Live exact-provider reviews are detached from the response and serialized by a profile review lock. Historical provider=all backfill is coalesced. Missing or non-project receipt cwd routes to user automation; registered-project identity routes to the relevant project automation root.

Managed skills and Hermes native discovery

TraceDecay remains the canonical owner of agent-managed skills. Hermes receives a native plugin overlay:

<hermes-profile>/plugins/tracedecay/
├── skills/tracedecay/SKILL.md
└── skills/agent-managed/<skill-id>/SKILL.md

Hermes is now a native overlay target. Install and plugin refresh export active managed skills into the TraceDecay-owned plugin package. Registration discovers bundled and generated skills through stock Hermes's native qualified skill system, producing names such as tracedecay:<skill-id>. Bundled skills win name collisions, names are validated, discovery order is deterministic, and incomplete packages are ignored safely.

TraceDecay never writes agent-managed output into host-owned user skill directories. Uninstall removes the overlay only when the TraceDecay manifest proves ownership. User-authored Hermes skills remain untouched.

Managed-skill transaction and concurrency hardening

The managed-skill store now uses durable directory transactions rather than best-effort multi-file replacement.

Key properties:

• async-safe lock acquisition (no await while holding the store mutex);
• one transaction covers complete skill directories and related metadata;
• durable journal writes and fsync before commit;
• roll-forward recovery after interruption;
• duplicate/self-merge rejection;
• create-only behavior for blind saves;
• locked export/snapshot reads;
• archived job skills excluded from active exports;
• post-commit ledger failures do not roll back an already durable skill commit;
• failure-path cleanup removes partial stages, backups, and temporary journals;
• safe uninstall remains ownership-manifest guarded.

Skill consolidation remains staged, checksum-guarded, provenance-preserving, and reversible. Pinned skills and user-authored content remain protected. Autonomous memory mutation does not imply autonomous destructive skill archival.

Multiple agents can safely read/write TraceDecay memory, sessions, automation ledgers, and managed-skill state through these locks/transactions. Agents editing the same source working tree still need separate worktrees or strict non-overlapping file ownership; TraceDecay does not silently merge overlapping Git edits.

Progressive tool discovery

TraceDecay registers non-core tools under toolset="tracedecay". Stock Hermes can defer them behind its progressive tool_search, tool_describe, and tool_call bridge.

The stock integration proves:

1. TraceDecay tools exist in the raw registered catalog.
2. Forced progressive assembly removes them from the initial visible tool list.
3. Hermes exposes its progressive bridge tools.
4. Semantic tool search finds TraceDecay tools.
5. Tool description returns their schemas.
6. Bridge invocation dispatches the underlying TraceDecay tool and arguments.

Only live-ingest LCM verbs that require Hermes to forward the in-memory message list are capability-gated. Normal code graph, session search, memory-provider, and compatible LCM operations participate in standard discovery. Agent-visible schemas remove internal scope-routing knobs while the raw MCP/CLI schemas remain complete.

Install, generated-plugin, and test isolation fixes

The generated-plugin harness now pins both HOME and TRACEDECAY_DATA_DIR to its throwaway profile. Previously a real exported profile root could make an isolated Hermes install wait on the live profile's managed-skill consolidation lock or inspect live skills. This was a harness isolation bug, not a production config double-save deadlock.

Hermes generated schemas are tested separately from agent-visible registered schemas: raw schemas retain public memory_scope/storage_scope, while registration deep-copies and removes internal Hermes routing knobs.

Transcript tests that change HOME, USERPROFILE, profile DB paths, or HERMES_HOME now share the profile-environment lock. This prevents parallel provider tests from redirecting one another's storage.

Hermes configuration after rollout

The intended TraceDecay-only Hermes integration is:

memory:
  provider: tracedecay
  memory_enabled: true
  user_profile_enabled: true

context:
  engine: tracedecay

TraceDecay is the selected memory provider and context engine. Hermes's user/profile memory behavior remains enabled but is served through TraceDecay rather than disabled. The earlier PR text that showed both booleans as false was obsolete and incorrect.

Validation

Final settled-diff validation performed locally:

• cargo fmt --all -- --check: passed
• git diff --check: passed
• cargo check --all-targets: passed
• cargo clippy --all-targets -- -D warnings: passed
• generated Hermes plugin checks: 38/38 passed
• stock upstream Hermes end-to-end: 24/24 passed
• stock progressive tool discovery: passed
• stock two-project memory isolation: passed
• stock plugin list/doctor checks: passed
• Hermes suite: 65/65 passed
• agent integration suite: 492/492 passed
• automation runner suite: 159/159 passed
• core CLI suite: 200/200 passed
• MCP suite (serial stability run): 377/377 passed
• memory suite: 52/52 passed
• storage suite: 292/292 passed
• transcript ingestion suite (serial stability run): 110 passed, 1 intentional ignore
• conventional commit validation: passed

The final Linux CI audit also removed a second-boundary race from the combined-review scheduler test: the interval assertion now supplies no artificial fresh-activity watermark, so crossing a wall-clock second cannot change the expected decision.

The stock validation used a throwaway HOME/profile, two initialized throwaway projects, and upstream Hermes ref 732a9ffc572ad2703fbd25cc8a21c9f3f9c10d69.

CI and release rollout

After CI is green, this PR will be merged. Then the release-plz update/release PR will be merged, the released binary installed, all supported integrations refreshed, and the Hermes gateway restarted so new sessions load the new plugin module and configuration.

Live rollout verification covers:

1. project Hermes session writes only to its project shard;
2. project subdirectory resolves to its registered root;
3. untethered Hermes session writes to user memory/user sessions;
4. two parallel Hermes sessions retain independent routes;
5. separate worktrees resolve correctly;
6. successful completed-turn receipts schedule review once;
7. autonomous curation performs validated mutation;
8. preview APIs remain read-only;
9. managed skills appear through qualified Hermes native discovery;
10. progressive tool search discovers and dispatches TraceDecay tools;
11. Codex/Cursor/Claude/Kiro projectless ingestion respects the project/user boundary.

Database recovery of any previously preserved malformed local stores is handled separately and conservatively: keep DB/WAL/SHM/sentinel recovery sets together, stop writers, recover into new candidates, compare integrity/counts, and replace only after verification. This PR never deletes the preserved recovery set.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 4281500

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f26952062d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".