Skip to content

ci: add MCP interface diff workflow for Everything server#3260

Open
SamMorrowDrums wants to merge 4 commits into
modelcontextprotocol:mainfrom
SamMorrowDrums:add-everything-server-mcp-diff-workflow
Open

ci: add MCP interface diff workflow for Everything server#3260
SamMorrowDrums wants to merge 4 commits into
modelcontextprotocol:mainfrom
SamMorrowDrums:add-everything-server-mcp-diff-workflow

Conversation

@SamMorrowDrums
Copy link
Copy Markdown

Summary

Adds a GitHub Actions workflow that tracks public interface changes to the Everything MCP server. This provides clear visibility into how the reference implementation evolves over time—useful for SDK compliance validation, catching unintended regressions, and reviewing PRs that modify the server interface.

Motivation

The Everything server is the reference implementation that exercises all MCP protocol features. Changes to its public interface (tools, resources, prompts, capabilities) often signal:

  • New protocol features being demonstrated
  • SDK updates affecting server behavior
  • Potential breaking changes

Currently there's no automated way to see these changes at a glance. This workflow surfaces them directly in the GitHub Actions summary.

Related discussion: modelcontextprotocol/inspector#1034 - Interest in using mcp-server-diff for SDK compliance testing with the Everything server.

What the workflow does

  1. Triggers on PRs and pushes affecting src/everything/
  2. Compares current branch against merge-base (PRs) or previous commit (pushes)
  3. Generates a diff report showing changes to tools, resources, prompts, and capabilities
  4. Supports manual workflow_dispatch for comparing any two refs (commits, tags, branches)

Example output

Here's a real diff between commit 737ce98 and the current HEAD of the Everything server:

--- base/initialize.json
+++ target/initialize.json

+ capabilities.tasks: {"cancel":{},"list":{},"requests":{"tools":{"call":{}}}}
--- base/tools.json
+++ target/tools.json

+ tools[simulate-research-query]: {"description":"Simulates a deep research operation..."}

This clearly shows the new Tasks capability and the simulate-research-query tool that were added.

Real-world example

See the MCP diff action in use on github/github-mcp-server - this workflow run shows interface changes from the "Generate server instructions in Inventory" PR.

Workflow features

  • Minimal permissions (contents: read only)
  • Pinned action versions with SHA references
  • Path filtering - only runs when Everything server changes
  • Manual trigger - compare any refs via workflow_dispatch
  • Clear summary - results appear directly in GitHub Actions UI

Testing

The workflow uses SamMorrowDrums/mcp-server-diff which is also available as a CLI:

# Try it locally
npx mcp-server-diff@latest -b "node dist/index.js stdio" -t "node dist/index.js stdio" -o markdown

Adds a GitHub Actions workflow that tracks public interface changes
to the Everything MCP server using mcp-server-diff.

Features:
- Runs on PRs and pushes affecting src/everything/
- Auto-compares against merge-base (PRs) or previous state (pushes)
- Manual workflow_dispatch for comparing any two refs
- Generates diff reports showing tool, resource, prompt, and capability changes

This helps catch unintended interface changes and provides clear
visibility into how the reference server evolves over time.

Related: modelcontextprotocol/inspector#1034
Composite actions don't support working-directory at step level.
Use full paths in commands instead.
@SamMorrowDrums
Copy link
Copy Markdown
Author

SamMorrowDrums commented Feb 2, 2026

@cliffhall what do you think I'd need to do, or who do you think I'd need to ping to get this over the line (or get a definitive no)?

As mentioned offer is open to:

  • make a trusted maintainer admin of the repo
  • hand the repo to mcp org

https://discord.com/channels/1358869848138059966/1466125436932198483/1466211900160348394

@cliffhall cliffhall closed this Mar 19, 2026
@cliffhall cliffhall reopened this Mar 19, 2026
@cliffhall cliffhall added enhancement New feature or request server-everything Reference implementation for the Everything MCP server - src/everything labels Apr 20, 2026
@cliffhall
Copy link
Copy Markdown
Member

@claude review

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented Apr 21, 2026

Claude finished @cliffhall's task in 1m 9s —— View job


PR Review: ci: add MCP interface diff workflow for Everything server

  • Gather context and understand the request
  • Read the workflow file
  • Analyze for correctness, security, and best practices
  • Post review feedback

Summary

The workflow is clean, focused, and follows most GitHub Actions best practices. The main considerations are around third-party action trust and a few minor improvements. Overall it's a reasonable addition if maintainers are comfortable taking a dependency on the external action.


Security

Permissionscontents: read only. Minimal and correct for what this workflow does.

Pinned SHAs — Both actions are pinned to commit SHAs, which is the right approach:

  • actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 (v4.2.2) ✅
  • SamMorrowDrums/mcp-server-diff@a5555e85d68eaa014a334ae7d12b73787f2c49cc (v2.3.5) ✅

Third-party action — the main concern: The action SamMorrowDrums/mcp-server-diff is authored by the same person submitting this PR. This is worth calling out explicitly for maintainers:

  • It's not under the modelcontextprotocol or actions org, meaning this repo takes an ongoing maintenance dependency on an external project
  • The SHA pin is good supply-chain hygiene, but updating requires manual tracking
  • Maintainers should review the action source before merging to understand what code runs in your CI environment
  • If this action becomes unmaintained or the author transfers/deletes the repo, the workflow breaks

This doesn't make it a hard blocker, but it's the key judgment call: is the visibility benefit worth an external runtime dependency on a non-org action? The PR author has offered to transfer the repo to the MCP org, which would resolve this concern entirely.


Correctness

Checkout fetch-depth: 0 — Correct. The diff action needs full history to find merge-base.

compare_ref empty string behavior (line 52): The behavior when compare_ref is '' is entirely delegated to the external action — the workflow has no fallback or documentation for what happens. The PR description says it uses merge-base for PRs and previous commit for pushes, but this is opaque from reading the workflow alone. A comment here would help future maintainers:

# When empty, mcp-server-diff uses merge-base for PRs and HEAD~1 for pushes
compare_ref: ${{ github.event.inputs.compare_ref || '' }}

start_command path (line 51): node src/everything/dist/index.js stdio runs relative to the checkout root. This is correct, but it's worth noting this path is hard-coded and would need updating if the server ever moves.

workflow_dispatch target_ref label (line 25): The description says "defaults to current branch if empty" which is accurate — the checkout step falls back to github.ref. Clear enough.


Minor Issues

No concurrency group — Without this, multiple pushes to a PR branch will queue duplicate runs rather than cancelling the superseded one. Fix this →:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

setup_node: 'true' as string (line 47): Minor — passing 'true' as a quoted string rather than a boolean is fine for YAML action inputs (they're always strings), but it's slightly unusual. No real issue here.

if: always() on summary step (line 56): Good practice — ensures the summary context is appended even on failure.


Overall Assessment

The workflow is well-structured and serves a legitimate purpose. The gate on merging should be:

  1. Maintainers should review SamMorrowDrums/mcp-server-diff source before accepting this dependency
  2. Consider whether transferring the action repo to the MCP org is the right path (as the author offered), which would make this more maintainable long-term

If those concerns are addressed, the main code change is solid. The concurrency group is a nice-to-have improvement that could be added in a follow-up or in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request server-everything Reference implementation for the Everything MCP server - src/everything

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants