Skip to content

feat(core): support per-session runtime model switching#2097

Open
Buktal wants to merge 1 commit into
agentscope-ai:mainfrom
Buktal:feat/per-session-model-switching
Open

feat(core): support per-session runtime model switching#2097
Buktal wants to merge 1 commit into
agentscope-ai:mainfrom
Buktal:feat/per-session-model-switching

Conversation

@Buktal

@Buktal Buktal commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds per-session runtime model switching for a single (singleton-scoped) HarnessAgent, so one agent instance can serve sessions that need different models — including different apiKeys — without redeploying or running one agent per tenant. This unlocks cluster deployment and multi-tenant isolation.

Closes #2085.

Approach

Mirrors the existing Agent#setPermissionMode runtime-switch pattern: persist a per-session pointer in AgentState, resolve it on each call.

  • The model id (not the model/apiKey) is persisted in AgentState (model_id). It rides along with the rest of the session state across the cluster (saveAgentState + activateSlotForContext reload), so a session lands on the same model on any node.
  • On every call, ReActAgent.getModel(ctx) resolves the id through ModelRegistry.namedModels (registered at startup, consistent across nodes). A null/blank id, or one that fails to resolve, falls back to the agent's default model.
  • Dedicated compaction / memory-flush models are still honored when configured (CompactionConfig.getModel() / MemoryConfig.model()); the per-session model is used only when no dedicated one is set.

Because Model is stateless (unlike PermissionEngine, which accumulates ASK rules), this is a strict subset of setPermissionMode: no per-slot cache rebuild is needed — resolution is fresh on each call. The apiKey never touches AgentState.

API

agent.setModel(userId, sessionId, modelId);  // persist id for the session
agent.setModel(ctx, modelId);                // via RuntimeContext
agent.clearModel(userId, sessionId);         // revert to default
Model m = agent.getModel(ctx);               // effective model for the call

Changes

File Change
AgentState persisted modelId field (model_id), builder, JSON round-trip
ReActAgent setModel / getModel / clearModel; getModel(ctx) resolves the session id via ModelRegistry with default fallback; modelForCall wraps with the configured fallback
HarnessAgent delegates for setModel / getModel / clearModel; build wires dedicated-vs-per-session model for compaction and flush
CompactionMiddleware prefer the dedicated model, else the per-session model
MemoryFlushMiddleware prefer the dedicated flush model, else the per-session model

Tests

  • ReActAgentPerSessionStateTest: per-session isolation, cluster-drift reload, clear → revert, unresolvable-id → default fallback, end-to-end call() routing through the per-session model
  • AgentStateTest: model_id JSON round-trip, old JSON missing the field → null, blank → null normalization
  • MemoryFlushMiddlewareTriggerTest: existing suite still green after the effective-model change

Notes

  • No breaking changes to existing model resolution; the default path is unchanged when no session id is set.
  • Public API has Javadoc; dedicated compaction/memory models still take precedence over the per-session model.

…i#2085)

Allow a single HarnessAgent instance to switch the model per session at
runtime, mirroring Agent#setPermissionMode. The session's model id is
persisted in AgentState (so it survives cluster drift) and resolved
through ModelRegistry.namedModels on each call; a null/blank id falls
back to the agent's default model. Models are stateless, so-unlike
permission mode-no per-slot cache rebuild is needed.

- AgentState: add persisted modelId field (json: model_id)
- ReActAgent: add setModel/getModel/clearModel; getModel(ctx) resolves
  the session id via ModelRegistry, falling back to the default on
  miss or resolution failure
- HarnessAgent: expose setModel/getModel/clearModel delegates
- CompactionMiddleware/MemoryFlushMiddleware: prefer the configured
  dedicated model, otherwise use the per-session model
- Tests: per-session isolation, cluster drift, clear, unresolvable
  fallback, end-to-end call routing, AgentState round-trip
@Buktal Buktal requested a review from a team July 10, 2026 03:38

@oss-maintainer oss-maintainer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Adds per-session runtime model switching for singleton-scoped HarnessAgent, enabling multi-tenant isolation without redeployment. Mirrors the existing setPermissionMode pattern.

Findings

  • [Info] AgentState.javamodelId field with JSON round-trip and backward-compatible deserialization (missing field → null). Clean.
  • [Info] ReActAgent.java:348-364 — Resolution via ModelRegistry.namedMaps.get(modelId) with fallback to default is correct. apiKey never touches AgentState — good security practice.
  • [Info] CompactionMiddleware.java / MemoryFlushMiddleware.java — Dedicated models still take priority over per-session model. Correct precedence.
  • [Info] Tests are comprehensive: per-session isolation, cluster drift reload, clear→revert, unresolvable-id fallback, end-to-end call routing.

Verdict

Well-designed feature with thorough test coverage. The approach of persisting only the model id (not the model instance or apiKey) is the right call for cluster safety. LGTM.


Automated review by github-manager

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: HarnessAgent 单实例时,如何动态切换模型?好像不支持

2 participants