Cut /tools latency: fix annotation N+1, cache WorkOS + tools.list#361
Open
RhysSullivan wants to merge 3 commits intomainfrom
Open
Cut /tools latency: fix annotation N+1, cache WorkOS + tools.list#361RhysSullivan wants to merge 3 commits intomainfrom
RhysSullivan wants to merge 3 commits intomainfrom
Conversation
openapi/graphql resolveAnnotations ran one listOperationsBySource per distinct scope in the toolRows. Store the owning scopeId on each StoredOperation instead and issue a single query, partitioning by op.scopeId so shadowed tools still resolve against the right binding. Also push executor.tools.list's sourceId filter down to storage so `/sources/:id/tools` stops pulling every row in the scope stack.
Every protected request authenticates twice — once in lookupOrgForRequest to choose the org, once in the OrgAuth middleware to populate AuthContext. Each uncached call re-runs the WorkOS refresh POST when the access token is expired. Wrap authenticateSealedSession in Cache.make keyed by the sealed session string so both callers share a single lookup per (sessionData, ~10s window).
tools.search loads the full tool list on every call to score against it. The MCP session DO survives the whole CLI session, so repeated searches share one DB fetch. Wrap executor.tools.list in a plain Map keyed by JSON.stringify(filter); DO idle timeout (5m) caps staleness. Subsequent searches drop from ~1.5s p50 to ~0ms.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
executor-cloud | 388f532 | Apr 21 2026, 06:40 AM |
@executor/sdk
@executor/plugin-file-secrets
@executor/plugin-google-discovery
@executor/plugin-graphql
@executor/plugin-keychain
@executor/plugin-mcp
@executor/plugin-oauth2
@executor/plugin-onepassword
@executor/plugin-openapi
@executor/plugin-workos-vault
executor
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three targeted latency fixes for the MCP /
/sources/:id/toolshot path. Measured end-to-end on/api/scopes/<org>/sources/<src>/tools: 5.91s → 1.03s; repeattools.searchin an MCP session: ~1.5s p50 → ~0ms.tools: collapse annotation N+1 + push sourceId filter to DB — openapi/graphql
resolveAnnotationsran onelistOperationsBySourceper distinct scope across the tool rows (N queries per annotate). Stores the owningscopeIdon eachStoredOperationand partitions a single query's result byop.scopeId, keeping shadowed tools resolving against the right binding. Also pushesexecutor.tools.list'ssourceIdfilter to storage so/sources/:id/toolsstops pulling every row in the scope stack and filtering client-side.workos: cache sealed-session auth for 10s within a request — every protected request authenticates twice (
lookupOrgForRequest+OrgAuthmiddleware). Each cache miss re-runs the WorkOS refresh POST when the access token is expired.Cache.makekeyed by sealed session collapses the two into one lookup within a ~10s window.mcp-session: cache
executor.tools.liston the DO —tools.searchscans the full list on every call; a single MCP session DO sees many searches. PlainMap<JSON.stringify(filter), Tool[]>in the DO closure, no TTL (DO idle timeout = 5m caps staleness). Cache state lands on theexecutor.tools.list.cachedspan ascache.state: hit|miss.Test plan
bunx tsc --noEmit -p apps/cloud/tsconfig.jsoncleanexecutor.tools.list.cachedshowshiton second+tools.searchin same session (0ms) vsmisson first (2.2s populate)/sources/cloudflare_api/toolsdropped from 5.91s → 1.03s post-deploy