feat: add Hermes Agent as a first-class backend and transcript source#114
feat: add Hermes Agent as a first-class backend and transcript source#114fgscaglioni wants to merge 6 commits into
Conversation
This adds full support for Nous Research's Hermes Agent across all three
SkillOpt layers — core model backend, sleep-cycle backend, and transcript
harvesting.
Core model layer (skillopt/model/):
- Register the Hermes CLI as a chat backend (hermes_chat) alongside the
existing Claude, Codex, Qwen, and MiniMax backends.
- All chat entry points (optimizer, target, messages, deployment) route
through the Hermes adapter.
Sleep-cycle backend (skillopt_sleep/backend.py):
- Add HermesBackend extending CliBackend, driving hermes --profile
<name> chat -Q -q for attempt / judge / reflect calls.
- CLI output is filtered to strip notices, warnings, and traceback
debris so the pipeline sees only the model response.
- Registered under get_backend("hermes") with aliases hermes_chat and
hermes_cli.
Transcript harvesting (skillopt_sleep/harvest_hermes.py):
- New harvest source reads Hermes session data from ~/.hermes/state.db
(SQLite), building SessionDigest objects from sessions + messages.
- Skips engine-internal sessions (temp dirs matching skillopt_sleep_*)
so the mine step only sees real user sessions.
- Supports the standard harvest contract: scope, since, limit, project.
Config additions (skillopt_sleep/config.py):
- memory_filename — controls the project memory file name.
Defaults to "CLAUDE.md" for backward compatibility.
Set to "AGENTS.md" for Codex and Hermes.
- hermes_home — path to Hermes state directory, defaults to
~/.hermes, overridable via $HERMES_HOME.
Tests:
- 7 new tests in TestHermesBackendCli covering backend registration,
command construction, error capture, output filtering, and env vars.
Backward compatible — defaults unchanged, all existing backends intact.
|
@microsoft-github-policy-service agree |
|
Thanks for the substantial Hermes integration. Supporting another agent/backend can improve SkillOpt, and we would like to preserve the work under your authorship, but the current PR is too broad and has several correctness issues that block merge. Confirmed issues:
We recommend splitting this into three reviewable PRs: core model backend, Sleep CLI backend, and transcript harvester. Each should include contract tests and one opt-in real Hermes smoke test. Please ensure backend invocations disable unintended tools/plugins/MCP when processing harvested untrusted text. We are not rejecting the feature direction; we are asking for a rebase and a narrower, test-backed design so it can be merged safely while keeping your contribution attribution. |
Yif-Yang
left a comment
There was a problem hiding this comment.
Requesting changes based on the backend-contract, routing, token-accounting, error-surfacing, harvesting, and test gaps detailed in the maintainer comment above. Please rebase and narrow or split the implementation before re-review.
…at_backend - Add hermes -> hermes_chat alias so normalize_backend_name() resolves correctly - Add hermes_chat default model in _BACKEND_DEFAULT_MODELS - Include hermes_chat in is_optimizer_chat_backend() and is_target_chat_backend() - Add hermes_chat to validation sets in set_optimizer_backend / set_target_backend Addresses Yif-Yang review: is_*_chat_backend now returns true for hermes_chat.
…er setters, message API contracts - Use local _hermes_tracker (TokenTracker) instead of global tracker from common.py to prevent token double-counting with Claude/OpenAI - Make target/optimizer profiles mutable via set_target_deployment / set_optimizer_deployment; setters now actually control which profile chat_target / chat_optimizer use - chat_target_messages / chat_optimizer_messages now respect: * retries parameter (not hardcoded 3) * return_message=True returns CompatAssistantMessage, not str * tools/tool_choice serialized into the prompt preamble - _call_hermes retries on non-zero exit with exponential backoff - subprocess.run wraps exception in RuntimeError instead of silent empty string Addresses Yif-Yang review: token accounting, deployment setters, message/tool contract.
26 tests covering: - Backend alias resolution and default models - set_target/optimizer_backend acceptance (ValueError boundaries) - is_*_chat_backend recognition - Routing dispatch (chat_target, chat_optimizer, chat_with_deployment) - Token tracker isolation (no double-count with Claude) - Message API contracts: * retries parameter respected * return_message=True returns CompatAssistantMessage * tools/tool_choice serialized into prompt * chat_messages_with_deployment uses custom profile - Deployment setters (set_target/optimizer_deployment) - Edge cases (env vars, non-zero exit, routing when not selected) - One opt-in smoke test (pytest.mark.slow)
ffaa1f1 to
524b864
Compare
|
Thanks for the thorough review, Yif-Yang. I've split the implementation into three branches as suggested and will open separate PRs shortly. |
This adds full support for Nous Research's Hermes Agent across all three SkillOpt layers — core model backend, sleep-cycle backend, and transcript harvesting.
Core model layer (skillopt/model/):
Sleep-cycle backend (skillopt_sleep/backend.py):
Transcript harvesting (skillopt_sleep/harvest_hermes.py):
Config additions (skillopt_sleep/config.py):
Tests:
Backward compatible — defaults unchanged, all existing backends intact.