Add conformance-server + importable conformance runner (addresses #674)#726
Open
qchuchu wants to merge 5 commits into
Open
Add conformance-server + importable conformance runner (addresses #674)#726qchuchu wants to merge 5 commits into
qchuchu wants to merge 5 commits into
Conversation
A WPT-style host-conformance test runner for the MCP Apps spec (2026-01-26): ships a ui:// page that renders inside the host's sandboxed iframe, drives the postMessage/JSON-RPC bridge, asserts host behaviour against the spec, and reports PASS/FAIL right in the iframe. First step toward modelcontextprotocol#674 (public MCP Apps platform tests). Mirrors the basic-server-react layout/toolchain and is registered alongside the other examples in both E2E specs and both publish workflows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…typed TestSuite SDK: new subpaths `@modelcontextprotocol/ext-apps/conformance` (Host interface, abstract Playwright BrowserHost, Runner → SubtestResult[]) and isomorphic `/conformance/protocol`; playwright declared as an optional peer dependency. examples/conformance-server: rewritten on the typed capability protocol (window.__mcpConformance pull model) — split harness (assert/host-gateway/registry/ channel), 8 starter tests spanning automatic/operator/interaction/fallback, and a host-neutral implementer guide + BrowserHost subclass template. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #693 (that PR became unreopenable after a rebase + force-push; this is the same branch, rebased onto latest
main).Summary
Two parts, both under the MCP Apps conformance umbrella (a step toward #674):
examples/conformance-server— aui://TestSuite that renders inside the host's sandboxed iframe, drives the postMessage/JSON-RPC bridge, and asserts host behaviour against the spec (2026-01-26). Rebuilt on a typed capability protocol: a test emits aCapabilityRequestand awaits the result over awindow.__mcpConformancepull channel. Split harness (assert/host-gateway/registryengine/channel), 8 starter tests spanning automatic / operator-DOM / agent-interaction / capability-fallback, and a host-neutral implementer guide.An importable conformance runner — new SDK subpaths so any host can conformance-test itself (dev-only):
@modelcontextprotocol/ext-apps/conformance—Hostinterface, an abstract PlaywrightBrowserHost(subclass it for your host), and aRunnerthat returnsSubtestResult[].@modelcontextprotocol/ext-apps/conformance/protocol— the isomorphic protocol types (imported by the app view and the runner alike).playwrightis declared as an optional peer dependency (kept external from the browser bundle).Product-specific driver bits (named-host adapters, results/video/report generators) intentionally stay out of the SDK.
Testing
npm run build(SDK) +npm run --workspace examples/conformance-server build: clean.npm run prettier: clean.servers.spec.tspasses for the conformance server (screenshot unchanged).🤖 Generated with Claude Code