Complete Modern client-facing dispatch: listen + pagination

docs/arch/03-transport-architecture.md

@@ -677,20 +677,31 @@ path for server->client messages.
   send-on-closed-channel panic if a concurrent broadcast/dispatch races
   eviction/shutdown.
 
-**Forward-compatibility note**: the 2026-07-28 (Modern) MCP revision (see
-`mcp.MCPVersionModern` and `pkg/mcp/revision.go`) introduces
-`subscriptions/listen`, a POST method whose response stream stays open for
-server-pushed subscription events. `serverStreamRegistry` is a reasonable
-decoupled fan-out seam to build a future Modern handler for this method on top
-of, but it is **not** a drop-in reuse. Per the 2026-07-28 spec, real adaptation is
-required: `subscriptions/listen` is a long-lived POST, not a GET; Modern has no
-sessions, so streams must be keyed per-subscription-id rather than per session
-ID; delivery requires per-notification-type AND per-URI opt-in filtering, not
-blanket fan-out; an initial `notifications/subscriptions/acknowledged` must be
-sent; and deliveries must be tagged with `io.modelcontextprotocol/subscriptionId`.
-`dispatcher_streams.go` is intentionally kept transport-agnostic (no HTTP
-types) so that this adaptation, when it happens, does not also require
-rewriting the underlying fan-out primitives.
+**Relationship to Modern `subscriptions/listen`**: the 2026-07-28 (Modern)
+revision (see `mcp.MCPVersionModern` and `pkg/mcp/revision.go`) replaces the
+standalone GET with `subscriptions/listen`, a POST method whose response stream
+stays open for server-pushed subscription events.
+
+vMCP now **serves** that method on its client edge
+(`pkg/vmcp/server/modern_subscriptions.go`), and deliberately does **not** use
+`serverStreamRegistry`. The reason is that the handler honors no subscriptions:
+the honored set is intersected against the capabilities `server/discover`
+advertises, every push flag there is false, so the acknowledged set is always
+empty and the stream terminates immediately. With nothing to fan out, wiring in a
+fan-out registry from another package would add an abstraction with zero
+consumers. What the handler does implement is the wire contract — a long-lived
+POST rather than a GET, keyed per-subscription-id (the listen request's own
+JSON-RPC id) rather than per session ID since Modern has no sessions, an initial
+`notifications/subscriptions/acknowledged`, and
+`io.modelcontextprotocol/subscriptionId` tagging.
+
+`serverStreamRegistry` remains the sensible seam for the piece still missing:
+actual **delivery** of notifications on a Modern listen stream (#5743). That work
+needs per-notification-type AND per-URI opt-in filtering rather than blanket
+fan-out, and per `schema/draft/schema.ts` must tag *every* notification with the
+subscription id, not just the acknowledgement. `dispatcher_streams.go` is
+therefore still intentionally kept transport-agnostic (no HTTP types), so that
+when delivery lands it does not also require rewriting the fan-out primitives.
 
 ## Error Handling
 

docs/arch/10-virtual-mcp-architecture.md

@@ -136,6 +136,19 @@ When backends expose tools with the same name, vMCP resolves the conflict using
 | **priority** | First backend in priority order wins, others hidden |
 | **manual** | Explicit mapping for each conflict |
 
+**Conflict resolution covers tools only.** Resources, resource templates, and
+prompts are concatenated across backends with no de-duplication
+(`default_aggregator.go`, "no conflict resolution for these yet"), so
+`Resource.URI`, `ResourceTemplate.URITemplate` and `Prompt.Name` can each repeat
+in the aggregated view — two backends both exposing `file:///README.md` is
+enough. Only `Tool.Name` is unique, because tool resolution keys a map.
+
+Anything that treats one of those fields as an identity must account for that.
+The Modern list paginator does: its cursor names a position within a run of equal
+keys rather than assuming keys are distinct, because a plain "resume after this
+key" scan silently dropped an item whenever a duplicate landed on a page boundary
+(see [Client-facing list pagination](#client-facing-list-pagination)).
+
 ### Tool Filtering
 
 Beyond conflict resolution, vMCP can filter which tools are exposed through allow/deny lists, renaming, and description overrides.
@@ -300,14 +313,53 @@ Beyond tools, vMCP aggregates and serves the full complement of MCP capabilities
 | Resource templates (`resources/templates/list`) | Yes | Templated reads route through the same `ReadResource` path; the router matches an expanded URI against the aggregated templates, and an exact template-string key routes to its backend; covered by the same `notifications/resources/list_changed` propagation as resources (MCP 2025-11-25 has no separate wire method for template changes) |
 | Prompts (`prompts/list`, `prompts/get`) | Yes | Served per-session; a backend's asynchronous `notifications/prompts/list_changed` propagates ADDITIONS (not removals) to already-registered sessions — see below |
 | Completions (`completion/complete`) | Yes | A `ref/prompt` routes via the prompts table; a `ref/resource` carries the URI-template string per the spec and routes via the resource-templates table (exact template-string key, with exact-resource and template-expansion fallbacks). Unroutable refs return an empty result (lenient completion); admission denial returns an error |
-| Resource subscriptions (`resources/subscribe`, `resources/unsubscribe`) | Ack-level only | vMCP accepts and records the subscription (after session-binding and resource-admission checks) but does **not** yet forward backend `notifications/resources/updated` — see the limitation below |
+| Resource subscriptions (`resources/subscribe`, `resources/unsubscribe`) | Ack-level only | Legacy edge. vMCP accepts and records the subscription (after session-binding and resource-admission checks) but does **not** yet forward backend `notifications/resources/updated` — see the limitation below |
+| Subscriptions (`subscriptions/listen`) | Ack-level only | Modern (2026-07-28) edge, the revision's only server→client push channel. Answered with an SSE stream carrying the mandatory `notifications/subscriptions/acknowledged` and then a terminating result, both tagged `io.modelcontextprotocol/subscriptionId` and keyed by the listen request's JSON-RPC id (Modern has no sessions). The honored set is intersected against the advertised capabilities, all of whose push flags are false, so it is always **empty** and the stream closes immediately rather than idling. Serving it is what lets a go-sdk v1.7 client complete `Connect` at all — see below |
+
+The four Modern list verbs (`tools/list`, `resources/list`, `resources/templates/list`, `prompts/list`) **paginate**, emitting `nextCursor` while items remain and capping a page at 1000 items to match the page size the SDK applies on the Legacy path. See [Client-facing list pagination](#client-facing-list-pagination).
 
 The completion handler is a single global handler installed via `WithCompletionHandler`, so it recovers the session from the SDK request context rather than a per-session closure. Setting it makes the shim auto-advertise the `completions` capability at initialize.
 
 ### Subscription limitation (ack-level)
 
 vMCP advertises `resources.subscribe: true` and answers `resources/subscribe` / `resources/unsubscribe` at **ack level**: the request is accepted (enforcing session binding and validating the URI is an advertised, admitted resource), and go-sdk records the subscription. vMCP does **not** currently propagate backend `notifications/resources/updated` to the subscribed client — doing so requires persistent per-session backend connections, which is out of scope. Clients that subscribe will receive a success ack but no update stream yet.
 
+### Client-facing list pagination
+
+On the Legacy edge the SDK's session-scoped feature store splits list results into
+pages. `dispatchModern` bypasses the SDK, so the Modern edge paginates itself
+(`pkg/vmcp/server/modern_pagination.go`).
+
+Modern has no sessions, so a cursor may not denote server-held iteration state.
+The draft pagination page makes cursors **opaque to clients** ("MUST treat cursors
+as opaque tokens"), which is precisely what allows the server to encode position
+*into* the token instead of remembering it. So the cursor is self-describing:
+base64url over a small JSON payload naming the list kind, the last key delivered,
+and how many items sharing that key have already been sent.
+
+Three properties worth knowing:
+
+- **Ordering.** Keyset paging needs a deterministic total order, and the
+  aggregator's fan-out order is not stable between calls, so Modern list results
+  are sorted by the item's key (`Tool.Name`, `Resource.URI`,
+  `ResourceTemplate.URITemplate`, `Prompt.Name`). Legacy ordering is unchanged.
+- **Duplicate keys are tolerated, not assumed away.** Only tool names are unique
+  (see [Conflict Resolution](#conflict-resolution)); resource URIs, template
+  strings and prompt names can repeat. The cursor therefore resumes *within* a run
+  of equal keys, because a plain "resume after this key" scan skipped every copy
+  and permanently dropped items whose key collided at a page boundary.
+- **End of results omits `nextCursor` entirely.** The draft states that "an empty
+  string is a valid cursor and thus MUST NOT be treated as the end of results", so
+  emitting `""` would make a conformant client re-request and loop on page one.
+
+The cursor encodes a position in the **aggregated** ordering and never names a
+backend, so adding or removing a backend cannot invalidate one. This is unrelated
+to the aggregator's *upstream* cursor-following (#5851), which is vMCP acting as a
+client walking a backend's pages.
+
+An invalid cursor — malformed, over-length, or minted for a different list verb —
+is rejected with `-32602`, per the draft's error-handling rule.
+
 ### Tools/resources/prompts list_changed propagation (#5748, #5969)
 
 Unlike the per-call backend client (`pkg/vmcp/client`), the **persistent**
@@ -338,8 +390,17 @@ above for how that revision is resolved and cached. This is correct, not a gap i
 `initialize` and `Mcp-Session-Id`, so there is no Legacy-shaped persistent
 connection to hold and no standalone GET stream a Modern backend could push
 on. Modern's own server-push mechanism is `subscriptions/listen` (see
-[Transport Architecture](03-transport-architecture.md)), which vMCP does not
-implement yet — so `list_changed` propagation is currently Legacy-only.
+[Transport Architecture](03-transport-architecture.md)). vMCP **serves** that
+method on its client edge (`pkg/vmcp/server/modern_subscriptions.go`), but only
+at acknowledgement level: it computes the honored subscription set by
+intersecting the client's request against the capabilities `server/discover`
+advertises, and since every push-related flag there is deliberately false
+(`newModernCapabilities`), the honored set is always empty and no notification is
+ever pushed. So `list_changed` **delivery** remains Legacy-only, on both edges —
+what the Modern client edge gained is a conformant, explicitly-empty answer
+instead of a `-32601` that tore the client's connection down. Real Modern
+delivery is tracked in #5743 and requires vMCP to start advertising a push
+capability first.
 
 The sink is built once per session, at registration (`pkg/vmcp/server`'s
 `buildListChangedSink`), closing over the SDK `ClientSession`, the session ID,

pkg/vmcp/server/modern_dispatch.go

@@ -97,34 +97,58 @@ func (s *Server) dispatchModern(w http.ResponseWriter, r *http.Request, parsed *
 		s.dispatchModernPromptGet(ctx, w, parsed, identity)
 	case "completion/complete":
 		s.dispatchModernComplete(ctx, w, parsed, identity)
+	case methodSubscriptionsListen:
+		// Ungated, same bucket as the list verbs and discover -- see
+		// dispatchModernSubscriptionsListen for why, and for the one future
+		// change that would require gating it.
+		s.dispatchModernSubscriptionsListen(ctx, w, parsed, identity)
 	case "ping":
-		// ping is deliberately ungated (unauthenticated liveness, same bucket
-		// as initialize -- no Check*) and carries NEITHER resultType NOR
-		// _meta.serverInfo on the wire: the SDK's ping handler returns
-		// emptyResult, and both annotateServerInfo and setCompleteResultType
-		// early-return/no-op for it (go-sdk server.go:1929-1945,1992). Do not
-		// route this through the envelope builders above -- a bare {} is the
-		// correct, spec-matching result.
+		// ping does NOT exist in 2026-07-28: `ping` appears nowhere in
+		// schema/draft/schema.ts, and go-sdk lists it among the methods removed
+		// for this revision (server.go:1880), answering -32601. So answering it
+		// at all is deliberate LENIENCY toward a client that pings anyway, not
+		// spec conformance -- an earlier version of this comment claimed the
+		// latter, which was wrong twice over: the method is gone, and the bare {}
+		// below omits `resultType`, which schema.ts's Result MUSTs on every
+		// result ("Servers implementing this protocol version MUST include this
+		// field").
+		//
+		// Kept as-is rather than switched to -32601 because that is a behavior
+		// change to a pre-existing path, out of scope for the change that
+		// introduced this comment fix. It is inert in practice: a go-sdk Modern
+		// client never pings (startKeepalive sits past an early return), so
+		// nothing observable depends on either answer today. If you are here to
+		// tighten it, -32601 is the conformant answer.
+		//
+		// It is ungated (unauthenticated liveness, same bucket as initialize --
+		// no Check*). Do not route it through the envelope builders above: they
+		// would stamp resultType and _meta.serverInfo, which the Legacy SDK path
+		// does not do for ping either (annotateServerInfo and
+		// setCompleteResultType both early-return for it, go-sdk
+		// server.go:1929-1945,1992), so the bare {} at least keeps the two paths
+		// answering alike.
 		writeModernResult(w, parsed.ID, struct{}{})
 	default:
 		writeModernError(w, parsed.ID, jsonRPCCodeMethodNotFound, "method not found")
 	}
 }
 
 // The four list-dispatch helpers below (tools/list, resources/list,
-// resources/templates/list, prompts/list) always return the full
-// admission-filtered set from the matching core.List* and do not emit a
-// nextCursor: the Modern list-result envelopes carry no cursor field at all
-// (PaginatedResult.nextCursor is optional, so omitting it is spec-valid),
-// client-facing cursor pagination is unimplemented, and any cursor a Modern
-// client sends is ignored. This is unrelated to the aggregator's UPSTREAM
+// resources/templates/list, prompts/list) each page the full
+// admission-filtered set from the matching core.List* through paginateModern,
+// emitting a nextCursor while items remain. The cursor is stateless keyset
+// pagination over the aggregated ordering -- see modern_pagination.go for the
+// wire contract and why a self-describing cursor is what a sessionless
+// revision requires. This is unrelated to the aggregator's UPSTREAM
 // cursor-following for internal discovery (#5851); that's a different layer.
 //
 // A List*/Discover failure logs the full error server-side and returns a
 // generic -32603 message to the client (writeModernListError below): unlike
 // the call/read/get verbs, these errors come from aggregation and routing
 // plumbing (backend IDs, upstream addressing), and security.md forbids
-// leaking that detail to callers.
+// leaking that detail to callers. An invalid CURSOR is the exception: it is
+// caller input rather than a server-side fault, so writeModernPageError maps
+// it to -32602 instead.
 func (s *Server) dispatchModernToolsList(
 	ctx context.Context, w http.ResponseWriter, parsed *mcpparser.ParsedMCPRequest, identity *auth.Identity,
 ) {
@@ -133,7 +157,18 @@ func (s *Server) dispatchModernToolsList(
 		writeModernListError(ctx, w, parsed.ID, parsed.Method, err)
 		return
 	}
-	result, err := newModernToolsList(tools, s.config.Name, s.config.Version)
+	cursor, err := modernRequestCursor(parsed.Params)
+	if err != nil {
+		writeModernPageError(ctx, w, parsed.ID, parsed.Method, err)
+		return
+	}
+	page, next, err := paginateModern(tools, func(t vmcp.Tool) string { return t.Name },
+		cursorKindTools, cursor)
+	if err != nil {
+		writeModernPageError(ctx, w, parsed.ID, parsed.Method, err)
+		return
+	}
+	result, err := newModernToolsList(page, s.config.Name, s.config.Version, next)
 	if err != nil {
 		writeModernListError(ctx, w, parsed.ID, parsed.Method, err)
 		return
@@ -149,7 +184,18 @@ func (s *Server) dispatchModernResourcesList(
 		writeModernListError(ctx, w, parsed.ID, parsed.Method, err)
 		return
 	}
-	writeModernResult(w, parsed.ID, newModernResourcesList(resources, s.config.Name, s.config.Version))
+	cursor, err := modernRequestCursor(parsed.Params)
+	if err != nil {
+		writeModernPageError(ctx, w, parsed.ID, parsed.Method, err)
+		return
+	}
+	page, next, err := paginateModern(resources, func(r vmcp.Resource) string { return r.URI },
+		cursorKindResources, cursor)
+	if err != nil {
+		writeModernPageError(ctx, w, parsed.ID, parsed.Method, err)
+		return
+	}
+	writeModernResult(w, parsed.ID, newModernResourcesList(page, s.config.Name, s.config.Version, next))
 }
 
 func (s *Server) dispatchModernResourceTemplatesList(
@@ -160,7 +206,18 @@ func (s *Server) dispatchModernResourceTemplatesList(
 		writeModernListError(ctx, w, parsed.ID, parsed.Method, err)
 		return
 	}
-	writeModernResult(w, parsed.ID, newModernResourceTemplatesList(templates, s.config.Name, s.config.Version))
+	cursor, err := modernRequestCursor(parsed.Params)
+	if err != nil {
+		writeModernPageError(ctx, w, parsed.ID, parsed.Method, err)
+		return
+	}
+	page, next, err := paginateModern(templates, func(t vmcp.ResourceTemplate) string { return t.URITemplate },
+		cursorKindResourceTemplates, cursor)
+	if err != nil {
+		writeModernPageError(ctx, w, parsed.ID, parsed.Method, err)
+		return
+	}
+	writeModernResult(w, parsed.ID, newModernResourceTemplatesList(page, s.config.Name, s.config.Version, next))
 }
 
 func (s *Server) dispatchModernPromptsList(
@@ -171,7 +228,18 @@ func (s *Server) dispatchModernPromptsList(
 		writeModernListError(ctx, w, parsed.ID, parsed.Method, err)
 		return
 	}
-	writeModernResult(w, parsed.ID, newModernPromptsList(prompts, s.config.Name, s.config.Version))
+	cursor, err := modernRequestCursor(parsed.Params)
+	if err != nil {
+		writeModernPageError(ctx, w, parsed.ID, parsed.Method, err)
+		return
+	}
+	page, next, err := paginateModern(prompts, func(p vmcp.Prompt) string { return p.Name },
+		cursorKindPrompts, cursor)
+	if err != nil {
+		writeModernPageError(ctx, w, parsed.ID, parsed.Method, err)
+		return
+	}
+	writeModernResult(w, parsed.ID, newModernPromptsList(page, s.config.Name, s.config.Version, next))
 }
 
 // dispatchModernDiscover serves server/discover, Modern's replacement for
@@ -433,6 +501,21 @@ func writeModernListError(ctx context.Context, w http.ResponseWriter, id any, me
 	writeModernError(w, id, jsonRPCCodeInternalError, "internal error")
 }
 
+// writeModernPageError classifies a paginateModern failure. An invalid cursor is
+// bad caller input, so it gets -32602 -- matching the spec's "handle invalid
+// cursors gracefully" and go-sdk, which returns ErrInvalidParams for a cursor it
+// cannot decode. The message deliberately does not say WHY the cursor was
+// rejected: clients must treat cursors as opaque, so describing the internal
+// encoding would invite them to construct one. Anything else here is an encode
+// failure, i.e. a server-side fault, and falls through to -32603.
+func writeModernPageError(ctx context.Context, w http.ResponseWriter, id any, method string, err error) {
+	if errors.Is(err, errInvalidModernCursor) {
+		writeModernError(w, id, jsonRPCCodeInvalidParams, "invalid cursor")
+		return
+	}
+	writeModernListError(ctx, w, id, method, err)
+}
+
 // writeModernCallFailure classifies a POST-dispatch error from the three call
 // verbs (tools/call, resources/read, prompts/get), layering the mid-call
 // capability-refusal case on top of writeModernDispatchError's authz/generic