feat(core): support per-session runtime model switching#2097
Open
Buktal wants to merge 1 commit into
Open
Conversation
…i#2085) Allow a single HarnessAgent instance to switch the model per session at runtime, mirroring Agent#setPermissionMode. The session's model id is persisted in AgentState (so it survives cluster drift) and resolved through ModelRegistry.namedModels on each call; a null/blank id falls back to the agent's default model. Models are stateless, so-unlike permission mode-no per-slot cache rebuild is needed. - AgentState: add persisted modelId field (json: model_id) - ReActAgent: add setModel/getModel/clearModel; getModel(ctx) resolves the session id via ModelRegistry, falling back to the default on miss or resolution failure - HarnessAgent: expose setModel/getModel/clearModel delegates - CompactionMiddleware/MemoryFlushMiddleware: prefer the configured dedicated model, otherwise use the per-session model - Tests: per-session isolation, cluster drift, clear, unresolvable fallback, end-to-end call routing, AgentState round-trip
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
oss-maintainer
approved these changes
Jul 10, 2026
oss-maintainer
left a comment
Collaborator
There was a problem hiding this comment.
Summary
Adds per-session runtime model switching for singleton-scoped HarnessAgent, enabling multi-tenant isolation without redeployment. Mirrors the existing setPermissionMode pattern.
Findings
- [Info]
AgentState.java—modelIdfield with JSON round-trip and backward-compatible deserialization (missing field → null). Clean. - [Info]
ReActAgent.java:348-364— Resolution viaModelRegistry.namedMaps.get(modelId)with fallback to default is correct.apiKeynever touchesAgentState— good security practice. - [Info]
CompactionMiddleware.java/MemoryFlushMiddleware.java— Dedicated models still take priority over per-session model. Correct precedence. - [Info] Tests are comprehensive: per-session isolation, cluster drift reload, clear→revert, unresolvable-id fallback, end-to-end call routing.
Verdict
Well-designed feature with thorough test coverage. The approach of persisting only the model id (not the model instance or apiKey) is the right call for cluster safety. LGTM.
Automated review by github-manager
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds per-session runtime model switching for a single (singleton-scoped)
HarnessAgent, so one agent instance can serve sessions that need different models — including differentapiKeys — without redeploying or running one agent per tenant. This unlocks cluster deployment and multi-tenant isolation.Closes #2085.
Approach
Mirrors the existing
Agent#setPermissionModeruntime-switch pattern: persist a per-session pointer inAgentState, resolve it on each call.AgentState(model_id). It rides along with the rest of the session state across the cluster (saveAgentState+activateSlotForContextreload), so a session lands on the same model on any node.ReActAgent.getModel(ctx)resolves the id throughModelRegistry.namedModels(registered at startup, consistent across nodes). Anull/blank id, or one that fails to resolve, falls back to the agent's default model.CompactionConfig.getModel()/MemoryConfig.model()); the per-session model is used only when no dedicated one is set.Because
Modelis stateless (unlikePermissionEngine, which accumulates ASK rules), this is a strict subset ofsetPermissionMode: no per-slot cache rebuild is needed — resolution is fresh on each call. TheapiKeynever touchesAgentState.API
Changes
AgentStatemodelIdfield (model_id), builder, JSON round-tripReActAgentsetModel/getModel/clearModel;getModel(ctx)resolves the session id viaModelRegistrywith default fallback;modelForCallwraps with the configured fallbackHarnessAgentsetModel/getModel/clearModel; build wires dedicated-vs-per-session model for compaction and flushCompactionMiddlewareMemoryFlushMiddlewareTests
ReActAgentPerSessionStateTest: per-session isolation, cluster-drift reload, clear → revert, unresolvable-id → default fallback, end-to-endcall()routing through the per-session modelAgentStateTest:model_idJSON round-trip, old JSON missing the field →null, blank →nullnormalizationMemoryFlushMiddlewareTriggerTest: existing suite still green after the effective-model changeNotes