[BLOCKED] Remove the vMCP Modern stateless dispatch kill-switch - #6033
[BLOCKED] Remove the vMCP Modern stateless dispatch kill-switch#6033JAORMX wants to merge 1 commit into
Conversation
Refs #5959. Config.ModernDispatchEnabled (env TOOLHIVE_VMCP_MODERN_STATELESS, default off) was a temporary safety lever gating whether well-formed MCP 2026-07-28 ("Modern") stateless requests were served by classifyingHandler -> dispatchModern or fell through to the SDK path. Both of the issue's gates are now met: the dual-era conformance harness landed in #5837 and go-sdk v1.7 was adopted via #5993. Serve Modern unconditionally and delete the switch, its ServerConfig/Config plumbing, its env var, and the test helpers and specs that existed only to toggle it. This also removes #6009's -32022 UnsupportedVersionError refusal and its server/discover exemption. That refusal answered exactly one condition -- "vMCP supports this revision but has chosen not to serve it" -- which cannot arise once the switch is gone. Every genuine version-grounds rejection still happens, earlier and unchanged, in mcpparser.ClassifyRevision: a _meta protocolVersion naming any other version yields the same conformant -32022, so ClassifyRevision is now the single owner of that decision. server/discover no longer needs an exemption because nothing refuses it on version grounds; dispatchModern answers it via dispatchModernDiscover. authz_integration_test.go's buildCedarAuthzServer had to *set* the switch for TestIntegration_CedarAuthzDenial_ModernPath_IsAudited to reach dispatchModern's call gate at all -- a silently-vacuous test class that disappears along with the switch. KNOWN BLOCKER, do not merge as-is: vMCP's Modern dispatcher does not implement subscriptions/listen. Once server/discover is dispatched rather than passed through, it advertises supportedVersions ["2026-07-28","2025-11-25"], so a go-sdk v1.7 client negotiates Modern and then opens the subscriptions/listen stream that mcpcompat's Initialize implies (it installs list-changed handlers unconditionally). dispatchModern answers -32601 and the client tears the connection down, breaking every client that wants server->client traffic. See pkg/vmcp/session/factory.go's initOneBackend, which already records "its push channel is subscriptions/listen, which vMCP does not implement" for the backend edge.
CI confirms the blocker, and corrects one claim in the description above
Zero specs passed. The
The other six are This is not a flake, and the shape proves itWorth stating explicitly, because this repo has had several genuine flakes today (#6026, #6034, a
Correction to the descriptionThe description says the dual-era e2e suite "would not have caught this" because So the coverage gap is narrower than stated — the Everything else in the description stands, including the baseline control showing this is pre-existing rather than introduced here. |
The server->client forwarding integration tests (mid-call elicitation, sampling, progress, logging) exercise a surface that exists only on a Legacy (2025-11-25) session: the 2026-07-28 revision removed server-initiated requests, replacing them with client-polled multi-round retrieval (SEP-2322), and SEP-2577 deprecates sampling and logging outright. Today these tests land on Legacy only incidentally, because the Modern dispatch kill-switch (#5959) rejects the go-sdk client's Modern-first probe; once the switch is removed (#6033) the client negotiates Modern and all of them fail at connect. Pin the downstream test clients to Legacy explicitly via a transport-level RoundTripper that answers server/discover with the same -32022 rejection the kill-switch produces (mcpcompat cannot pin a protocol version, #5911), and pin the Modern-client contract for the same eliciting backend in a new integration test: an explicit JSON-RPC error naming the refused request, never a hang, a fabricated success, or an input_required envelope. Document the client-edge limitation and the intended future MRTR shape (#5743) in the vMCP architecture doc. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GvBj7nNWqDQ3rKYNeY6aam
dispatchModern's four list verbs returned the full aggregated set and never emitted nextCursor, so a client had no way to page and vMCP silently ignored any cursor it was sent. On the Legacy path the SDK's session-scoped feature store does this split; the Modern dispatcher bypasses the SDK, so it has to do it itself. Modern has no sessions, so a cursor may not denote server-held iteration state. The spec makes cursors opaque to clients -- they MUST NOT parse, modify, or persist them -- which is exactly what lets the server encode position INTO the token instead of remembering it. So the cursor carries the unique key of the last item delivered and the next page is the items sorted strictly above it: keyset pagination, stateless, and the same scheme go-sdk's own server uses. Aggregation across backends needs no per-backend positions. core.List* already returns one flat, conflict-resolved set whose keys are unique across backends, so the cursor encodes a position in the aggregated ordering and adding or removing a backend cannot invalidate it. Page size matches the SDK default (1000) so pages line up across revisions, and an undecodable or wrong-verb cursor is rejected as -32602 rather than being reinterpreted into a plausible but wrong page. The upstream cursor-following in pkg/vmcp/client and the session backend (#5851) is the opposite direction -- vMCP as client -- and is untouched. Refs #5959, #6033
The server->client forwarding integration tests (mid-call elicitation, sampling, progress, logging) exercise a surface that exists only on a Legacy (2025-11-25) session: the 2026-07-28 revision removed server-initiated requests, replacing them with client-polled multi-round retrieval (SEP-2322), and SEP-2577 deprecates sampling and logging outright. Today these tests land on Legacy only incidentally, because the Modern dispatch kill-switch (#5959) rejects the go-sdk client's Modern-first probe; once the switch is removed (#6033) the client negotiates Modern and all of them fail at connect. Pin the downstream test clients to Legacy explicitly via a transport-level RoundTripper that answers server/discover with the same -32022 rejection the kill-switch produces (mcpcompat cannot pin a protocol version, #5911), and pin the Modern-client contract for the same eliciting backend in a new integration test: an explicit JSON-RPC error naming the refused request, never a hang, a fabricated success, or an input_required envelope. Document the client-edge limitation and the intended future MRTR shape (#5743) in the vMCP architecture doc. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GvBj7nNWqDQ3rKYNeY6aam
MCP 2026-07-28 removed the standalone HTTP GET stream, so subscriptions/ listen is the revision's only server->client push channel. vMCP's Modern dispatcher had no case for it, which made vMCP unreachable for any go-sdk v1.7 client: Connect is Modern-first, and once server/discover succeeds it opens a listen stream whenever a list-changed handler is registered -- which mcpcompat's Initialize does unconditionally. The resulting -32601 failed Connect outright and tore the session down. The honored set is computed by intersecting the client's requested notification types against the capability advertisement server/discover publishes, now extracted as newModernCapabilities so the two answers cannot drift. Every push-related flag there is deliberately false, so the honored set is empty today and the acknowledgement says so explicitly. That is the spec's own mechanism for declaring unsupported notification types, not a silent no-op: nothing is ever reported as subscribed and then dropped. No push delivery exists yet (#5743); vMCP must start advertising a push capability before this stream can carry anything. Measured against the kill-switch removal in #6033: 20 regressions before, 6 after. The remaining 6 are unrelated to this channel -- five need server-initiated elicitation/sampling that 2026-07-28 removes outright, and one needs Modern list pagination. Refs #5959, #6033, #5743
dispatchModern's four list verbs returned the full aggregated set and never emitted nextCursor, so a client had no way to page and vMCP silently ignored any cursor it was sent. On the Legacy path the SDK's session-scoped feature store does this split; the Modern dispatcher bypasses the SDK, so it has to do it itself. Modern has no sessions, so a cursor may not denote server-held iteration state. The spec makes cursors opaque to clients -- they MUST NOT parse, modify, or persist them -- which is exactly what lets the server encode position INTO the token instead of remembering it. So the cursor carries the unique key of the last item delivered and the next page is the items sorted strictly above it: keyset pagination, stateless, and the same scheme go-sdk's own server uses. Aggregation across backends needs no per-backend positions. core.List* already returns one flat, conflict-resolved set whose keys are unique across backends, so the cursor encodes a position in the aggregated ordering and adding or removing a backend cannot invalidate it. Page size matches the SDK default (1000) so pages line up across revisions, and an undecodable or wrong-verb cursor is rejected as -32602 rather than being reinterpreted into a plausible but wrong page. The upstream cursor-following in pkg/vmcp/client and the session backend (#5851) is the opposite direction -- vMCP as client -- and is untouched. Refs #5959, #6033
Spec-conformance review found the pagination comment citing the 2025-11-25 page for a Modern implementation. The draft page is what 2026-07-28 ships and it differs in one substantive way: "Don't persist cursors across sessions" is replaced by "Don't make any determination based on cursor value other than whether a non-null value was provided (e.g. an empty string is a valid cursor and thus MUST NOT be treated as the end of results)". Cite the draft page as the rule and SEP-2567 as the reason for the deletion, by path rather than line since that SEP exists in two copies with different numbering. That amendment makes the existing omitempty on nextCursor load-bearing rather than incidental: emitting an empty cursor would have a conformant client re-request with it, which reads as "first page", looping forever on page one. Pinned explicitly now. Reject a cursor that is present but not a JSON string instead of silently reading it as "first page". The draft requires invalid cursors to yield -32602 and schema.ts classifies cursor values under invalid params, so a client with a serialization bug was getting a silent full-list restart in place of a diagnosable error. An empty-string cursor still means "first page", matching go-sdk, since vMCP never mints one. Stop attributing the subscriptionId key, request-id-as-identity, and acknowledgement ordering to the go-sdk: all three are normative in schema/draft/schema.ts. The comment as written invited a maintainer to weaken a MUST. Also restate the delivery-side MUST -- every notification on a listen stream carries the key -- at the #5743 hand-off, since no code here delivers notifications and it would otherwise be lost. Correct the ping comment, which claimed spec support that does not exist: ping was removed in 2026-07-28 and the bare {} also omits the MUSTed resultType. Behavior left unchanged as out of scope; the conformant answer is noted for whoever tightens it. Rename TestPaginateModern_DeliversEveryItemExactlyOnce to say "static set": the exactly-once guarantee holds only while the set does not change mid-walk, which is inherent to keyset paging and out of scope per SEP-2567. Refs #5959, #6033
Review found the paginator asserting a precondition that holds for one of its four callers. Only Tool.Name is unique -- ResolveToolConflicts keys a map -- while resources, resource templates and prompts are plain-appended across backends under "no conflict resolution for these yet". With a bare "resume strictly above lastKey" scan, a key shared at a page boundary made the scan skip every copy, so an item appeared on no page at all. Two honest backends both exposing file:///README.md was enough. A 1100-item corpus delivered 1097. The cursor now carries how many items sharing the last key were already sent, so a page can resume inside a run of equal keys. Items sharing a key are interchangeable for that purpose, which keeps the walk correct without a secondary sort key the generic signature cannot see. The comment claiming uniqueness is corrected rather than merged: it now states which callers can collide and that the paginator does not rely on uniqueness. The test corpus was the reason this shipped. Every case drew from a helper minting strictly unique keys, so the one precondition the code called load-bearing was the only thing untested. Collisions must also be placed by SORTED position -- a distinctively named duplicate key sorts away from the boundary and tests nothing, which a first attempt at the fixture did. Cap an inbound cursor before decoding it. A well-formed 6MB cursor decoded successfully at ~295ms of CPU on a verb that is not rate-limited, and a legitimate cursor is tens of bytes. Skip the capability fan-out in subscriptions/listen when nothing could be honored. core.Discover is un-rate-limited and, while no push capability is advertised, its result cannot change the answer -- so N no-op listens cost N fan-outs, now worse since list verbs page. Probing the ceiling first makes that O(1). This renders the fan-out error path unreachable, so its test is removed rather than left asserting a branch the code cannot enter. Assert the SSE framing. Replacing the blank-line event terminator with a single newline previously left the whole suite green -- including against a real client -- while a conformant parser would dispatch neither frame, so the ack-then-result contract was unverifiable. The helper now parses strictly, and the method name, acknowledgement name, subscriptionId key and jsonrpc field are asserted as literals; mutating any of them was silent. Add the negative capability test. The design rests on the advertisement honoring nothing, which was enforced by no test at all -- only by a runtime WARN that fires in production after the offending line ships. Now asserted across all sixteen presence combinations. Use http.ResponseController rather than a Flusher assertion: every ResponseWriter wrapper in the chain defines Flush unconditionally, so the assertion only established that a wrapper was present. Correct the comment that claimed otherwise, and the one claiming headers were committed on all four failure paths when three precede WriteHeader. Log the honored set by count, not by client-supplied URIs. Update docs/arch: subscriptions/listen is no longer unimplemented, serverStreamRegistry is the seam for delivery rather than for the handler, and both the pagination behaviour and the tools-only uniqueness property are now recorded. Refs #5959, #6033, #5743
Fold in the remaining review findings, including one the panel could not reach and two fixture blind spots. Set X-Accel-Buffering: no on the SSE response. The draft Streamable HTTP page states servers SHOULD send it so a reverse proxy does not accumulate events before forwarding, and this is the only SSE-emitting path in the codebase. The panel missed it because the draft moved transports.mdx into a directory, so every path it tried 404'd; the page is now at docs/specification/draft/basic/transports/streamable-http.mdx. Checking it also settles two open questions: the SSE event name is NOT mandated anywhere, and resumability is explicitly unsupported, which is why no id: field is emitted. Split frame marshalling out of the stream writer. Both marshals could previously fail on a path documented as "headers already committed" when nothing had been written at all, so the client got a bare 200 with an empty body for a request carrying an id. Building first means those failures are still reportable as -32603, and only a genuine mid-stream write failure is unreportable. Shuffle the corpus before paging in drainPages. Deleting the sort from paginateModern entirely used to fail exactly one subtest -- the only one whose input was unsorted -- so the deterministic total order the whole keyset scheme depends on was effectively unverified. The aggregator's real output is unordered anyway, so a shuffled input is also the realistic one. The same mutation now fails four test functions. Cover the remaining duplicate-key shapes, whose loss profiles differ sharply: a single duplicate at the boundary lost one item, one mid-page lost none, a run of 50 lost 50, and an all-identical corpus lost a whole page and returned an empty second page. The rule is that every item sharing the last key delivered on a page was dropped, so loss scaled with the run straddling the boundary -- and a duplicate away from a boundary was harmless, which is what made it easy to miss. Binary-search the resume point instead of scanning, making a full walk O(n log n) rather than O(n^2). The insertion point also gives the since-removed-key behaviour for free. Refs #5959, #6033, #5743
Summary
Config.ModernDispatchEnabled(envTOOLHIVE_VMCP_MODERN_STATELESS, default off) was a temporary safety lever gating whether well-formed MCP 2026-07-28 ("Modern") stateless requests are served byclassifyingHandler→dispatchModernor fall through to the SDK path. #5959 asks for its removal before GA.Both of the issue's stated gates are met — #5837's dual-era conformance harness landed, go-sdk v1.7 was adopted via #5993 — and the sequencing @jhrozek asked for on #6009 is satisfied. So this looked ready. It is not, and the reason is the substance of this PR.
The blocker
Removing the
server/discoverexemption is what breaks the product — not removing the-32022.That exemption was added in #6009 to keep version negotiation possible while the switch was off. It turns out to have been doing load-bearing work well beyond conformance: it is what steered Modern-first clients onto Legacy.
Connectis Modern-first: it POSTsserver/discoverwithMCP-Protocol-Version: 2026-07-28.supportedVersions→ client negotiates Legacy → everything works.dispatchModernDiscoveranswerssupportedVersions: ["2026-07-28","2025-11-25"](newModernDiscover,modern_envelope.go) → client negotiates Modern.mcpcompat'sInitializeinstalls the list-changed handlers unconditionally (client.installNotificationHandlers), and go-sdk opens the standalone stream whenever any handler is set → client POSTssubscriptions/listen.dispatchModern's method switch has nosubscriptions/listencase (modern_dispatch.go:80-110coverstools/*,resources/*,prompts/*,completion/complete,ping,server/discover) →-32601→ go-sdk treatsConnectas failed and tears the connection down.Observed wire trace:
This is pre-existing, not introduced here
Confirmed by control: with this change stashed, setting
ModernDispatchEnabled: trueon cleanmainreproduces identical failures. The kill-switch has been masking a hole that already existed. Please read the 20 regressions below as exposed, not caused.pkg/vmcp/session/factory.goalready documents this exact hole for the backend edge — "its push channel is subscriptions/listen, which vMCP does not implement". This exposes the same hole on the client edge.So #5959 has an unstated third gate: implement
subscriptions/listenindispatchModern, or otherwise stop advertising Modern to clients that need a push channel. That is a Modern push-channel design task, not kill-switch work, and probably wants its own issue under #5743.Suggested path, and one fix to avoid
Implement
subscriptions/listenindispatchModern(or decide deliberately how vMCP should present Modern to clients needing a push channel), then rebase this on top.Answering
subscriptions/listenwith a benign stub is not a fix. Modern clients would connect successfully and then receive a silently non-functional notification channel — no elicitation, no sampling, no progress, nolist_changed, and no error to indicate it. That is strictly worse than the loud failure above, and it is the "silent no-op" anti-pattern.claude/rules/go-style.mdcalls out by name. Flagging it because it is the obvious shortcut and it would be very hard to notice in review.Type of change
Test plan
task lint) — 0 issues aftertask lint-fix(3gcifailures first, from struct-field alignment left over where the field was deleted)go vet ./...— clean. Note the e2e files carry no build tag (justpackage e2e_test), so plain vet does cover them.task build— passtask test) — FAILS, see below20 top-level regressions, all one root cause, all absent from a clean-tree baseline (failure sets diffed with and without the change):
pkg/vmcp/server(5)TestForwarding_Elicitation_RealBackend,TestForwarding_Sampling_RealBackend,TestForwarding_Sampling_RealBackend_SessionIsolation,TestForwarding_Progress_RealBackend,TestForwarding_Logging_RealBackendtest/integration/vmcp(15)TestVMCPServer_TypeCoercion,…_TypeCoercion_NestedAndArrays,…_StructuredContent,…_StructuredContent_IntegerComparisonError,…_ConflictResolution,…_PassthroughHeaders,…_CompositeToolNonStringArguments,…_DefaultResults_{ConditionalSkip,ContinueOnError,NestedStructure},…_Telemetry_CompositeToolMetrics,…_TwoBoundaryAuth_HeaderInjection,TestRegression_Over1000Tools_CompleteSetReceived,TestRegression_SetSessionTools_FixedAtInitialize_NoMidSessionReconciliation,TestRegression_SlidingSessionTTL_TrafficKeepsSessionAliveFailing on baseline too, unrelated:
pkg/api,pkg/secrets/keyring,test/e2e, all 11cmd/thv-operator/test-integration/*, 2test/e2e/thv-operator/*.API Compatibility
v1beta1API.No CRD or
api/v1*type is touched. The kill-switch was env-only and was never wired to a CRD field, so nothing in the operator API surface changes.server.Config/ServerConfiglose a Go field and theTOOLHIVE_VMCP_MODERN_STATELESSenv var is removed.Does this introduce a user-facing change?
Yes — and it is the reason this is blocked. vMCP would serve MCP 2026-07-28 unconditionally, and the env var would no longer be honoured. In its current form that also makes vMCP unreachable for clients needing a server→client channel.
Special notes for reviewers
@jhrozek — this removes two things you added in #6009, and they deserve separate verdicts:
-32022UnsupportedVersionErrorremoval is sound. It answered exactly one condition — "vMCP supports this revision but has chosen not to serve it" — which cannot arise once the switch is gone. Every genuine version-grounds rejection still happens, earlier and unchanged, inmcpparser.ClassifyRevision.server/discoverexemption is the interesting part. It was documented as a conformance measure (so a client could still negotiate down), but in practice it was silently pinning Modern-first clients to Legacy — which is a much stronger role, and the thing that keeps vMCP working today. Your read on whether that was intentional would be valuable.Why the tempting middle option does not work. Keeping the exemption while dropping the
-32022leaves vMCP serving Modern verbs while denying Modern in its own capability probe — Modern dispatch becomes unreachable for any well-behaved client, which is worse than either end state.A coverage gap worth attention independently of this PR: the dual-era e2e suite would not have caught this.
e2e.RawMCPClientnever sends theserver/discoverprobe, so the suite's Modern coverage does not exercise the negotiation path a real SDK client takes.Silently-vacuous test removed along the way:
authz_integration_test.go'sbuildCedarAuthzServerhad to set the switch forTestIntegration_CedarAuthzDenial_ModernPath_IsAuditedto reachdispatchModern's call gate at all. That class of test — passing while never touching the path it names — disappears with the switch.Refs #5959
Generated with Claude Code