Skip to content

fix(mcp): stabilize authenticated tool discovery#5754

Open
j15z wants to merge 16 commits into
stagingfrom
fix/mcp-discovery-races
Open

fix(mcp): stabilize authenticated tool discovery#5754
j15z wants to merge 16 commits into
stagingfrom
fix/mcp-discovery-races

Conversation

@j15z

@j15z j15z commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Authenticated MCP servers now keep the newest valid discovery result instead of resurfacing stale tools, stale status, or an OAuth prompt from an older request. Static-header authorization failures remain header-auth failures, while OAuth servers that do not advertise dynamic client registration return an actionable 422 directing the user to configure a token.

Discovery publication is ordered end to end. Redis and memory cache adapters allocate a monotonic millisecond mutation token; conditional cache publication and the database lastToolsRefresh CAS use that exact same token, so retries cannot make the two layers choose different winners. A credential-safe SHA-256 discovery revision covers every connection-affecting field; after a status CAS race, discovery re-reads that revision so metadata-only edits keep valid live tools while connection changes invalidate stale results. Failure counters recompute through an optimistic JSONB CAS when a success races them, and a full cache clear invalidates existing mutation owners before deleting entries.

Mutation ownership is retried once. If ordering or an atomic cache transition remains unavailable, freshly fetched tools are returned to the caller but unordered cache/database state is not persisted. If another mutation supersedes the request, discovery reloads the winning positive-cache entry; stale-configuration results with no valid winner still fail closed. Explicit configuration and list-change invalidation remains best-effort during cache degradation.

Bulk discovery awaits each publication before returning, failures persist a zero tool count, and refresh consumes the discovery publication metadata. It re-reads current server metadata, reports unavailable or winner-cache results with the live tool count, and compares the published lastToolsRefresh token instead of a wall-clock timestamp before applying a failure. Raw upstream errors are replaced with allowlisted UI messages plus shared credential-free structural diagnostics.

Investigation confirmed @modelcontextprotocol/sdk 1.29 already captures Mcp-Session-Id from initialize and replays it on tools/list. The client records the request phase and whether a session id was present without logging the identifier or credentials.

Related: #5665 and #5595.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Other: ___________

Testing

  • DATABASE_URL=postgresql://postgres:postgres@localhost:5432/sim bun run --cwd apps/sim test -- lib/mcp 'app/api/mcp/servers/[id]/refresh/route.test.ts' — 357 tests passed across 19 files.
  • bun run check:api-validation:strict — passed with no boundary-policy drift.
  • Biome check and git diff --check — passed.
  • Direct GitHub Copilot MCP probes completed initialize and tools/list with 47 tools using both the normal and SSRF-pinned transports; the SDK transport carried the session id.
  • bun run --cwd apps/sim type-check currently reaches only unrelated shared-worktree dependency errors in fork audit events, recordAuditBatch, and @google-cloud/storage; no MCP TypeScript errors are reported.

Reviewer focus: the unified cache/database publication token, explicit discovery revision, status JSONB CAS loop, Redis conditional-mutation scripts, and degraded-cache response path are the highest-value concurrency review points.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Screenshots/Videos

N/A — API and backend behavior only.

@vercel

vercel Bot commented Jul 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 18, 2026 8:34am

Request Review

@cursor

cursor Bot commented Jul 18, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Large changes to MCP discovery concurrency (cache ordering + DB CAS), authentication classification, and refresh API behavior; mistakes could cause stale tools, wrong connection status, or auth mis-routing.

Overview
This PR hardens MCP tool discovery so concurrent refreshes, cache clears, and config edits no longer publish stale tools, status, or OAuth prompts. Discovery now uses monotonic cache mutation tokens (beginMutation / applyMutationIfCurrent on memory and Redis) aligned with a database lastToolsRefresh CAS, plus a discoveryRevision hash so connection-affecting changes invalidate results while metadata-only edits can still return live tools.

The server refresh API calls discoverServerToolsWithMetadata, re-reads persisted server rows instead of overwriting status itself, skips workflow sync when discovery is superseded, and surfaces unavailable / winner-cache states with live tool counts. Bulk discovery awaits publication per server; failures use allowlisted user messages and credential-safe structural logging via getMcpSafeErrorDiagnostics.

OAuth start returns 422 with a clear message when dynamic client registration is unsupported. McpClient treats UnauthorizedError as OAuth-only authorization-required; header auth gets “Authentication failed” without triggering OAuth flows. MCP outbound traffic uses createPinnedMcpFetch (HTTP/2 via optional allowH2 on pinned fetch). Cache invalidation on config/transport/auth changes and full workspace clears advances mutation owners before deletes.

Reviewed by Cursor Bugbot for commit b9fe771. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/lib/mcp/client.ts
@greptile-apps

greptile-apps Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR stabilizes authenticated MCP tool discovery and publication. The main changes are:

  • Orders cache and database updates with one monotonic mutation token.
  • Handles concurrent discovery, retries, cache clears, and configuration changes.
  • Uses a millisecond range for PostgreSQL timestamp comparisons.
  • Improves OAuth and static-header authentication errors.
  • Adds credential-safe MCP diagnostics and HTTP/2 support for pinned MCP requests.
  • Updates refresh reporting and adds concurrency-focused tests.

Confidence Score: 5/5

This looks safe to merge.

  • The half-open millisecond comparison preserves PostgreSQL sub-millisecond timestamps without accepting the next millisecond.
  • Memory and Redis adapters allocate strictly increasing mutation tokens.
  • Cache and database publication use the same token, with stale owners rejected.
  • No blocking issue remains in the updated timestamp-publication path.

Important Files Changed

Filename Overview
apps/sim/lib/mcp/service.ts Adds ordered discovery publication, timestamp-range comparisons, configuration revision checks, and degraded-cache handling.
apps/sim/lib/mcp/storage/memory-cache.ts Adds monotonic in-memory mutation ownership and invalidates active owners during cache clears.
apps/sim/lib/mcp/storage/redis-cache.ts Adds atomic Redis scripts for mutation allocation, conditional cache updates, and clear ordering.
apps/sim/app/api/mcp/servers/[id]/refresh/route.ts Uses discovery publication metadata and persisted refresh tokens when reporting refresh results.
apps/sim/lib/mcp/client.ts Separates static-header authorization failures from OAuth prompts and adds credential-safe diagnostics.
apps/sim/lib/mcp/pinned-fetch.ts Enables HTTP/2 for pinned and SSRF-guarded MCP requests.

Reviews (3): Last reviewed commit: "fix(mcp): acquire a fresh cache mutation..." | Re-trigger Greptile

Comment thread apps/sim/lib/mcp/service.ts Outdated
- preserve typed static-header authentication failures\n- order discovery publication by start time\n- atomically publish tools and failure cache state
Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/lib/mcp/service.ts
Match configuration generations within the JavaScript millisecond window so PostgreSQL microseconds do not suppress valid publication.
@j15z

j15z commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator Author

The database publication predicate needs a precision-safe configuration generation.

Addressed in 89dce25: both status-publication predicates now match the JavaScript millisecond window, accepting PostgreSQL sub-millisecond precision while still rejecting the next generation. A boundary regression covers the behavior.

Comment thread apps/sim/lib/mcp/service.ts
- fall back to best-effort cache writes when mutation ordering is unavailable\n- share bounded redacted MCP error diagnostics across client and service
Comment thread apps/sim/lib/mcp/service.ts Outdated
Retry cache mutation ownership before publishing discovery state, avoid unordered publication that older writers can supersede, and keep explicit invalidation best effort.
Comment thread apps/sim/lib/mcp/service.ts
Comment thread apps/sim/lib/mcp/storage/redis-cache.ts
Use timestamp-based mutation tokens for both cache ownership and database CAS ordering, and invalidate Redis mutation owners before deleting entries during a full clear.
Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/lib/mcp/service.ts Outdated
Recompute consecutive failures after status CAS conflicts, reload winning cached tools for superseded discoveries, and return live tools without unordered publication when cache ordering is unavailable.
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
Expose discovery publication metadata to refresh callers, report live tools during cache degradation, and compare lastToolsRefresh mutation tokens instead of wall clocks when preserving a newer success.
Comment thread apps/sim/lib/mcp/service.ts Outdated
Probe mutation ownership after database CAS misses, keep valid live tools for metadata-only edits, and invalidate ownership for transport and auth-type changes.
Comment thread apps/sim/lib/mcp/service.ts Outdated
Hash connection-affecting server fields and recheck the fresh revision after database CAS misses so metadata edits retain valid tools while URL, credential, and transport changes fail closed.
MCP servers are commonly behind HTTP/2 fronts (CDNs, cloud LBs), but the
SSRF-pinned undici Agent is HTTP/1.1-only unless it opts into h2 via ALPN.
Add an allowH2 option to createPinnedFetch (default false, so LLM-provider
callers are unchanged) and centralize the MCP h2 decision in one
createPinnedMcpFetch helper routed through the transport, the OAuth probe,
and the SSRF-guarded discovery/revocation fetch. Pinning is unaffected: the
pinned lookup forces every connection to the resolved IP regardless of the
negotiated protocol.
…lpers

- Delete the unused setIfCurrentMutation/deleteIfCurrentMutation cache
  methods (interface + memory + redis adapters) and their two Redis Lua
  scripts; production only uses applyMutationIfCurrent. Tests rewritten onto
  applyMutationIfCurrent (ordering coverage preserved) or dropped where they
  only exercised the removed Lua.
- Drop the vestigial fallback param from getDiscoveryFailureMessage (all
  callers coerced to 'Connection failed').
- Route isDynamicClientRegistrationUnsupported through getErrorMessage to
  match the module's error-inspection convention.
@waleedlatif1

Copy link
Copy Markdown
Collaborator

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator

@cursor review

Comment thread apps/sim/lib/mcp/service.ts
A retried discovery took mutation ownership once before the retry loop, so a
retry that succeeded after a concurrent clearCache published under a stale
ownership id, lost the CAS, and dropped otherwise-valid tools (empty
fail-closed result). Begin a fresh mutation per attempt so a retried result
publishes under a current id. Adds a regression test.
@waleedlatif1

Copy link
Copy Markdown
Collaborator

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit cdc3f46. Configure here.

j15z added 2 commits July 18, 2026 01:17
Reacquire cache mutation ownership for each discovery retry and cover cache invalidation during retry backoff.

Note: the full app suite has pre-existing failures in unrelated workflow, webhook, chat, and UI tests; the focused MCP suite passes 38/38.
Comment thread apps/sim/lib/mcp/service.ts
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
Publish cache invalidation tokens into database ordering and fail refresh closed when discovery is superseded.

Focused MCP suites pass 48/48 and API validation passes. Note: the full app suite retains the same 41 pre-existing failures in unrelated workflow, webhook, chat, and UI tests.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b9fe771. Configure here.

refreshedServer.lastConnected > discoveryStartedAt
refreshedServer?.lastToolsRefresh != null &&
(server.lastToolsRefresh == null ||
refreshedServer.lastToolsRefresh > server.lastToolsRefresh)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Invalidation hides refresh failures

Medium Severity

newerSuccessWonRace treats any newer lastToolsRefresh as a successful discovery, but markServerCacheInvalidated also advances that token while leaving connectionStatus as connected. A failed refresh racing list_changed or clearCache can therefore return connected with a null error and hide the real discovery failure.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit b9fe771. Configure here.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newerSuccessWonRace treats any newer lastToolsRefresh as a successful discovery, but markServerCacheInvalidated also advances that token while leaving connectionStatus as connected.”

This is valid: lastToolsRefresh now orders success, failure, OAuth, and invalidation, so a newer value alone cannot prove that a success won. This is also the third review cycle in the same publication-race class, so I’m pausing before another automatic patch.

Two bounded options:

  1. Require both a newer lastToolsRefresh and a newer lastConnected before preserving connected status. This uses existing fields; invalidation does not advance lastConnected.
  2. Persist an explicit publication outcome/generation and make the route read that. This is clearer long-term but requires a larger schema/API change.

My lean is option 1 for this PR: it closes the reported race without a migration, while retaining the mutation token as ordering rather than overloading it as success provenance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants