feat(js/plugins/mcp): Refactor to a client manager structure, with dynamic tool calling. #2884

ssbushi · 2025-05-07T19:58:39Z

Modified the current McpClient class to a Client manager class
GenkitMcpClient is now a dedicated single client-server connection.
Tools are now fetched dynamically to support adding/removing/disabling clients in the manager.

google-cla · 2025-05-07T19:59:34Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

mbleigh

One more high-level piece of food for thought: in the client manager context, I think we probably want to be very robust to misconfiguration errors. It's possible people will use this for dynamic, user-configured tools and when that happens we should recover gracefully as best we can.

So maybe in the client manager there should be an {error?: {message, status, details}} and if there's an error trying to connect to the server it is put in a disabled state, an error-level log is emitted, and the error is also saved with the server.

js/plugins/mcp/README.md

js/plugins/mcp/src/client/client.ts

js/plugins/mcp/package.json

js/plugins/mcp/src/client/client.ts

js/plugins/mcp/src/client/index.ts

mbleigh · 2025-05-10T16:00:40Z

Oh also I think we should put this work into a launch branch - I created launch/mcp-redux and made #2896 to squash my own work and create a chain of commits. Can you rebase onto the launch/mcp-redux as your merge target?

ssbushi · 2025-05-13T19:21:37Z

One more high-level piece of food for thought: in the client manager context, I think we probably want to be very robust to misconfiguration errors. It's possible people will use this for dynamic, user-configured tools and when that happens we should recover gracefully as best we can.

So maybe in the client manager there should be an {error?: {message, status, details}} and if there's an error trying to connect to the server it is put in a disabled state, an error-level log is emitted, and the error is also saved with the server.

I'll work on this in a new commit, updating the PR to reflect the changes so far.

ssbushi · 2025-05-13T19:31:07Z

js/plugins/mcp/README.md


-All MCP actions are namespaced under the name you supply, so a client called `filesystem` will register tools such as `filesystem/read_file`.
+<!-- TODO: Note about MCP prompts. -->


@mbleigh @pavelgj

How should we approach prompts with the newer dynamic API? Tools and resources can be wrapped as dynamic tools, but prompts should be treated differently, I suppose....

We don't have dynamic prompt support today and I'm wondering how to expose them in this new iteration of the plugin.

Should we also provide the user to install the MCP client as a plugin like we did for mcpClient({..}) and provide prompts there?

Let's think short-term / long-term.

Short-term, maybe we should essentially follow the API of ExecutablePrompt but not actually register them? So basically:

const mcpPrompt = mcpManager.prompt('some-prompt'); await mcpPrompt(input, options); await mcpPrompt.stream(input, options); await mcpPrompt.render(input, options);

and under the hood that's just rendering the prompt to MessageData[] and then either executing it, streaming it, or returning the rendered options?

Long-term I think we should have an MCP manager able to be used as a plugin with full dynamic registry integration.

I like the idea of bespoke prompt references. One thing I'd suggest for the short term is to use a serverName qualifer to narrow it down:

const mcpPrompt = mcpManager.prompt({serverName: 'my-mcp-server', prompt: 'some-prompt'});

We can add support for dynamic look-ups as we go forward and handle server resolution when there is not conflicts.

Long-term I think we should have an MCP manager able to be used as a plugin with full dynamic registry integration.

Agreed. We need to discuss more on how to support dynamic registrations via plugins.

Maybe mcpManager.prompt('server-name', 'some-prompt')?

Added prompt support. I had to leave the following as unimplemented:

ExecutablePrompt.render -- no way to render the prompt in MCP prompts (AFAIK)

ExecutablePrompt.asTool -- @pavelgj said this is unimplemented in Genkit today.

Not sure if my implementation is accurate. LMK if I made a blunder here.

js/plugins/mcp/README.md

js/plugins/mcp/package.json

Co-authored-by: Pavel Jbanov <pavelj@google.com>

…-redux

js/plugins/mcp/src/client/client.ts

js/core/package.json

js/plugins/mcp/README.md

mbleigh

This is mostly looking really good, thanks for all the work! I think probably the most important thing after this will be adding a good suite of tests to give us more confidence that it's robust and ready for an upgrade.

mbleigh · 2025-05-16T23:15:32Z

js/plugins/mcp/README.md

+    // Each key (e.g., 'fs', 'git') becomes a namespace for the server's tools.
+    fs: {
+        command: 'npx',
+        args: ['-y', '@modelcontextprotocol/server-everything', ...ALLOWED_DIRS],


I think you probably meant something other than server-everything here.

mbleigh · 2025-05-16T23:16:31Z

js/plugins/mcp/README.md

+        -   **`url`**: (string) The URL of a remote server to connect to using the SSE MCP transport.
+        -   **`serverWebsocketUrl`**: (string) The URL of a remote server to connect to using the WebSocket MCP transport.
+        -   **`transport`**: An existing MCP transport object for connecting to the server.
+
 Most MCP servers are built to run as spawned processes on the same machine using the `stdio` transport. When you supply the `serverProcess` option, you are specifying the command, arguments, and environment variables for spawning the server as a subprocess.


No serverProcess option anymore.

mbleigh · 2025-05-16T23:17:20Z

js/plugins/mcp/README.md

+### [Deprecated] `mcpClient()`
+
+Note: This method is deprecated, please use `createMcpClient` or `createMcpManager` instead.
+
+For simpler scenarios involving a single MCP server, or for backward compatibility, the legacy `mcpClient()` function is still available:

-### Using MCP Actions
+```ts
+import { mcpClient } from 'genkitx-mcp/legacy'; // Note the import path
+
+const legacyFilesystemClient = mcpClient({
+  name: 'filesystemLegacy',
+  serverProcess: { /* ... */ },
+});
+```
+This function takes similar options to a single server configuration within `createMcpManager` (e.g., `name`, `version`, `serverProcess`, `rawToolResponses`) and directly returns a Genkit plugin for that single client. It is recommended to use `createMcpManager` for new projects.


We can leave this out - keep the deprecated method around but it doesn't need to be in the README.

mbleigh · 2025-05-16T23:18:32Z

js/plugins/mcp/README.md

+
+The legacy `mcpServer()` function is deprecated; please use `createMcpServer()` instead.


Suggested change

The legacy `mcpServer()` function is deprecated; please use `createMcpServer()` instead.

mbleigh · 2025-05-16T23:26:35Z

js/plugins/mcp/src/client/client.ts

Thinking out loud: it would be great to build up some tests for this library, and maybe the easiest way we could do that is creating a custom transport that would let us have a "mock" MCP server.

mbleigh · 2025-05-17T00:03:11Z

js/plugins/mcp/src/client/manager.ts

+  ) {
+    this._clientStates[serverName] = { error };
+    logger.error(
+      `An error has occured while managing your MCP client '${serverName}'. The client may be disabled to avoid further issues. Please resolve the issue and reenable the client '${serverName}' to continue using its resources.`


Probably a warn level log here.

mbleigh · 2025-05-17T00:04:50Z

js/plugins/mcp/src/util/prompts.ts

+}
+
+/**
+ * Lookup all tools available in the server and register each as a Genkit tool.


Suggested change

* Lookup all tools available in the server and register each as a Genkit tool.

* Lookup all prompts available in the server and register each as a Genkit prompt.

mbleigh · 2025-05-17T00:05:46Z

js/plugins/mcp/src/util/resources.ts

+
+function listResourcesTool(
+  ai: Genkit,
+  client: any, // Use 'any' or let TS infer; removing specific type import


Why any? If it's a circular dependency issue, you can do import type without triggering circ dependencies.

mbleigh · 2025-05-17T00:09:58Z

js/plugins/mcp/src/util/resources.ts

I think resources need to be handled a bit differently in the Manager. I don't want there to be a tool per client to list and read resources - there should be a single tool at the manager level (perhaps with optional list of server names to include) for listing / reading across all connected servers.

mbleigh · 2025-05-17T00:47:32Z

js/plugins/mcp/package.json

Let's go ahead and change the package name to @genkit-ai/mcp so we don't forget to do it later. Should be safe since we're merging to a launch branch.

mbleigh and others added 7 commits April 28, 2025 18:28

Refactors MCP plugin to multi-server client implementation.

975d91a

updates

8bf443f

Merge branch 'main' into sb/mcp-redux

e8eaacb

WIP, rough working stuff

590107a

WIP, rough working stuff

b3d21c0

docs changes, small refactors

e5594f9

refactor reconnection logic

6b5752e

github-project-automation bot added this to Genkit Backlog May 7, 2025

github-actions bot added docs Improvements or additions to documentation js go tooling python Python config sample root labels May 7, 2025

ssbushi changed the base branch from mcp-redux to main May 7, 2025 19:59

ssbushi added 4 commits May 7, 2025 16:03

undo pack

6bb7f00

Merge branch 'main' into sb/mcp-redux

23b9d7d

more refining

cf7ce67

README updates

8eee9ff

ssbushi requested review from mbleigh and pavelgj May 8, 2025 16:41

ssbushi marked this pull request as ready for review May 8, 2025 16:41

mbleigh changed the base branch from main to launch/mcp-redux May 10, 2025 15:26

mbleigh reviewed May 10, 2025

View reviewed changes

ssbushi added 2 commits May 12, 2025 14:14

merge with launch/mcp-redux

f705656

WIP, applied feedback; update, test and push evals app

624fa05

ssbushi added 5 commits May 12, 2025 17:29

WIP, applied feedback; update, test and push evals app

0538aeb

update evals testapp

72abbd8

update controls

da27257

update docs

cc84f6d

more tweaks

587e278

ssbushi requested a review from mbleigh May 13, 2025 19:23

ssbushi commented May 13, 2025

View reviewed changes

pavelgj reviewed May 13, 2025

View reviewed changes

js/plugins/mcp/README.md Outdated Show resolved Hide resolved

pavelgj reviewed May 13, 2025

View reviewed changes

js/plugins/mcp/package.json Outdated Show resolved Hide resolved

ssbushi and others added 4 commits May 14, 2025 09:34

Update js/plugins/mcp/README.md

dd28c9b

Co-authored-by: Pavel Jbanov <pavelj@google.com>

Update js/plugins/mcp/package.json

e497118

Co-authored-by: Pavel Jbanov <pavelj@google.com>

updated deps

19bb189

Merge branch 'sb/mcp-redux' of github.com:firebase/genkit into sb/mcp…

2b2a38c

…-redux

ssbushi commented May 14, 2025

View reviewed changes

js/plugins/mcp/src/client/client.ts Outdated Show resolved Hide resolved

ssbushi added 4 commits May 15, 2025 10:29

add prompts, error handling/logging

1027808

more fixes

e92d435

format

cfdace0

format

5983a99

ssbushi requested a review from pavelgj May 15, 2025 17:44

pavelgj reviewed May 16, 2025

View reviewed changes

js/core/package.json Outdated Show resolved Hide resolved

pavelgj reviewed May 16, 2025

View reviewed changes

js/plugins/mcp/README.md Outdated Show resolved Hide resolved

pavelgj reviewed May 16, 2025

View reviewed changes

js/plugins/mcp/README.md Outdated Show resolved Hide resolved

ssbushi added 2 commits May 16, 2025 10:43

undo zod version

5712a72

readme update

63b5df8

mbleigh reviewed May 17, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(js/plugins/mcp): Refactor to a client manager structure, with dynamic tool calling. #2884

feat(js/plugins/mcp): Refactor to a client manager structure, with dynamic tool calling. #2884

ssbushi commented May 7, 2025 •

edited

Loading

google-cla bot commented May 7, 2025

mbleigh left a comment

mbleigh commented May 10, 2025

ssbushi commented May 13, 2025

ssbushi May 13, 2025

mbleigh May 13, 2025

ssbushi May 14, 2025

mbleigh May 14, 2025

ssbushi May 15, 2025

mbleigh left a comment

mbleigh May 16, 2025

mbleigh May 16, 2025

mbleigh May 16, 2025

mbleigh May 16, 2025

mbleigh May 16, 2025

mbleigh May 17, 2025

mbleigh May 17, 2025

mbleigh May 17, 2025

mbleigh May 17, 2025

mbleigh May 17, 2025


		All MCP actions are namespaced under the name you supply, so a client called `filesystem` will register tools such as `filesystem/read_file`.
		<!-- TODO: Note about MCP prompts. -->


		The legacy `mcpServer()` function is deprecated; please use `createMcpServer()` instead.

	* Lookup all tools available in the server and register each as a Genkit tool.
	* Lookup all prompts available in the server and register each as a Genkit prompt.

feat(js/plugins/mcp): Refactor to a client manager structure, with dynamic tool calling. #2884

Are you sure you want to change the base?

feat(js/plugins/mcp): Refactor to a client manager structure, with dynamic tool calling. #2884

Conversation

ssbushi commented May 7, 2025 • edited Loading

google-cla bot commented May 7, 2025

mbleigh left a comment

Choose a reason for hiding this comment

mbleigh commented May 10, 2025

ssbushi commented May 13, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbleigh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ssbushi commented May 7, 2025 •

edited

Loading