feat(integrations): Facade tool surface (run/snapshot/screenshot) - #2665
Conversation
|
There was a problem hiding this comment.
All reported issues were addressed across 10 files
Architecture diagram
sequenceDiagram
participant Client as MCP Client
participant Server as stagehand-facade stdio bin
participant Config as ConfigFromEnv
participant Tools as StagehandFacadeTools
participant Stagehand as Stagehand
participant Runtime as PlaywrightCompat Runtime
participant Browser as Browserbase/Local
Note over Client,Browser: NEW: Playwright-facade tool surface over MCP stdio
Client->>Server: initialize
Server-->>Client: serverInfo stagehand-facade@4.0.0
Client->>Server: tools/list
Note over Server: custom ListTools handler returns pinned FACADE_TOOLS contract verbatim (no browser launch required)
Server-->>Client: run / snapshot / screenshot wire JSON schemas
Note over Client,Browser: First tools/call lazily creates resources (once per process)
Client->>Server: tools/call run {code}
Server->>Server: CodeModeRunInputSchema.parse (code XOR actions)
Server->>Config: stagehandFacadeConfigFromEnv()
Config-->>Server: STAGEHAND_BROWSER + BROWSERBASE_API_KEY/PROJECT_ID + model config
alt STAGEHAND_BROWSER=browserbase (or API key present)
Server->>Browser: browserbase.launch({apiKey, projectId?})
else local (default headless)
Server->>Browser: localBrowser.launch({headless: true})
end
Server->>Stagehand: Stagehand.create({browser, model...})
Stagehand-->>Server: stagehand
Server->>Tools: new StagehandFacadeTools(stagehand) — serialized queue starts
Tools->>Tools: enqueue → runNow → activePage()
Tools->>Stagehand: experimentalBatch(PRELUDE + user code + EPILOGUE, 60s timeout)
Stagehand->>Runtime: createPlaywrightCompatRuntime(batchStagehand) serialized into page
Runtime-->>Stagehand: compat page / context / browser proxies
Stagehand-->>Tools: RunEnvelope { value | executionError }
alt success
Tools-->>Client: result value
else user code threw
Tools->>Tools: rethrow executionError as Error with original stack
end
Server->>Server: sanitizeErrorMessage (redacts sk-... / apiKey / token / signingKey)
Server-->>Client: text result or isError text
Note over Client,Browser: snapshot hydrates per-page id map
Client->>Server: tools/call snapshot {includeIframes}
Server->>Tools: SnapshotInputSchema.parse → enqueue → snapshotNow
Tools->>Tools: page.snapshot() and store {url, xpathById} keyed by page.pageId
Tools-->>Client: formatted tree with bracketed [a-b] ids
Note over Client,Browser: run {actions} resolves ids against latest snapshot
Client->>Server: tools/call run {actions:[{op:'click',id:'0-19'}]}
Server->>Tools: RefActionSchema parse → enqueue → runActionsNow
Tools->>Tools: activePage() + lookup snapshot state for that page
alt no snapshot for this page
Tools-->>Client: NO_HYDRATED_SNAPSHOT_ERROR
else page.url() != snapshot.url
Tools-->>Client: NAVIGATED_SNAPSHOT_ERROR
else id missing from xpathById
Tools-->>Client: staleSnapshotIdError(id)
else all ids hydrate to xpath= selectors
Tools->>Stagehand: experimentalBatch(ACTION_RUNNER_SOURCE, hydrated actions)
Stagehand-->>Tools: {completed}
Tools-->>Client: {completed, url}
end
Note over Client,Browser: screenshot returns image content block
Client->>Server: tools/call screenshot {type, fullPage, quality}
Server->>Tools: ScreenshotInputSchema.parse → enqueue → screenshotNow
Tools->>Tools: page.screenshot() → base64 data + mimeType (png/jpeg)
Server-->>Client: text 'Screenshot captured.' + image content block
Note over Server,Browser: Shutdown on SIGINT/SIGTERM/stdin EOF closes in order
Server->>Stagehand: stagehand.close()
Server->>Browser: browser.close() → exit 0 (1 if unclean)
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
|
Update: dropped the top-level The Browserbase smoke via the Eve example surfaced that AI-SDK-based MCP clients (Eve, Vercel AI SDK) reject tool input schemas with a top-level Re-smoke after the fix (Eve → HTTP bridge → stdio facade → Browserbase, model gpt-5-mini): proxy confirmed Follow-up (tracked for the deepagents consolidated pass on #2653): mirror the same wire-schema relaxation in the Python server so both hosts advertise identical bytes. |
There was a problem hiding this comment.
All reported issues were addressed across 6 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
|
Added commit 28c29f2 exposing the facade library surface via a |
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
…screenshot) with stdio MCP host Promotes the playwrightCompatRuntime and the run/snapshot/screenshot tool contract from the facade experiment into packages/integrations as product code: contract.ts carries the reference tool definitions verbatim (schemas, descriptions, error strings), tools.ts adapts StagehandFacadeTools to a live Stagehand handle with per-page snapshot state and a serialized call queue, and a new stagehand-facade stdio bin exposes the three tools over MCP with screenshots as image content blocks.
AI-SDK-based MCP clients (Eve, Vercel AI SDK) reject tool input schemas with a top-level oneOf, failing every run call client-side before it reaches the server. The code/actions exclusivity stays in the tool description and is enforced at runtime by CodeModeRunInputSchema.
Native consumers (Eve defineTool, Vercel AI SDK tools) need direct access to StagehandFacadeTools and the pinned contract, not just the stdio bin.
- widen credential redaction (Browserbase, Google, bearer tokens) - bound shutdown against an in-flight launch (5s race) - surface unsupported model names as StagehandFacadeConfigError - only forward integral jpeg quality to page.screenshot - fail waitForOutput fast when the host exits before ready
One system prompt shared by every host example instead of three hand-authored variants.
…mples packages/integrations/ becomes a grouping directory: core/ is the @browserbasehq/stagehand-integrations package, and integration examples (deepagents, eve, vercel-ai) sit beside it. Fixes eve dev's snapshot EINVAL: eve copies a workspace dependency's package root, which previously contained the eve example itself.
9010314 to
6ab64c8
Compare
The protocol commits on this branch updated the schema without running the Python generator; CI's generate.py --check caught the drift.
The Gemini 3.6 Flash protocol change on this branch made the Go-embedded extension stale (extensionpack --check and the wheel-smoke build both fail on it). The docs sdk-reference parity test times out at its 5s default on cold PR runners (observed 5090ms) — give the multi-language surface extraction a real budget.
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
There was a problem hiding this comment.
1 issue found across 2 files (changes from recent commits).
Confidence score: 2/5
- In
packages/integrations/core/src/facade/config.ts, the default local stdio facade now launches headed Chromium, which can fail on headless/container hosts without$DISPLAY, creating a concrete startup regression for non-interactive environments. Restore a headless default (or gate headed mode behind an explicit environment/control flag) to de-risk runtime failures.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/integrations/core/src/facade/config.ts">
<violation number="1" location="packages/integrations/core/src/facade/config.ts:82">
P1: The default local stdio facade no longer starts in headless/container hosts: it launches headed Chromium, which fails without `$DISPLAY`. Keep the non-interactive default headless (or add an explicit environment-controlled opt-in for headed debugging).</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
| ...(browserbaseProjectId ? { projectId: browserbaseProjectId } : {}), | ||
| }, | ||
| } | ||
| : { type: "local", launchOptions: { headless: false } }, |
There was a problem hiding this comment.
P1: The default local stdio facade no longer starts in headless/container hosts: it launches headed Chromium, which fails without $DISPLAY. Keep the non-interactive default headless (or add an explicit environment-controlled opt-in for headed debugging).
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/integrations/core/src/facade/config.ts, line 82:
<comment>The default local stdio facade no longer starts in headless/container hosts: it launches headed Chromium, which fails without `$DISPLAY`. Keep the non-interactive default headless (or add an explicit environment-controlled opt-in for headed debugging).</comment>
<file context>
@@ -79,7 +79,7 @@ export function stagehandFacadeConfigFromEnv(
},
}
- : { type: "local", launchOptions: { headless: true } },
+ : { type: "local", launchOptions: { headless: false } },
stagehand,
};
</file context>
CrewAI example for the Stagehand facade tool surface, consuming it as a standard MCP server over stdio. - **Tools:** `run` / `snapshot` / `screenshot` — the shared facade contract; agent backstory is the canonical facade instructions text, pinned to the TS constant by a drift test (skips when lifted out of the repo). - **Transport:** `crewai-tools` `MCPServerAdapter` + stdio, spawning the built `stagehand-facade` bin with an env allowlist (`STAGEHAND_*`/`BROWSERBASE_*` only). Screenshots are saved to files (CrewAI's adapter drops MCP image blocks — documented in the README). - **Run:** `packages/integrations/crewai/README.md` (uv project, mirrors the deepagents conventions). Tests run via `uv run pytest` locally and in review; they are not wired into CI (no Python job covers this path). Verified: contract + adapter + drift tests pass locally; Browserbase smoke passed (heading cited with snapshot ID, clean child shutdown). Stacked on #2665.
Mastra example for the Stagehand facade tool surface, consuming it as a
standard MCP server over stdio.
- **Tools:** `run` / `snapshot` / `screenshot` — the shared facade
contract; system prompt from `FACADE_AGENT_INSTRUCTIONS`.
- **Transport:** Mastra `MCPClient` + stdio, spawning the workspace
`stagehand-facade` bin with an env allowlist
(`STAGEHAND_*`/`BROWSERBASE_*` only); `client.disconnect()` in
`finally`.
- **Run:** `packages/integrations/mastra/README.md`.
Verified: contract test in CI; Browserbase smoke passed ("Heading (from
snapshot [0-19]): Example Domain", no zombie processes).
Stacked on #2665.
Vercel AI SDK example for the Stagehand facade tool surface, consuming it as a standard MCP server over stdio. - **Tools:** `run` / `snapshot` / `screenshot` — the shared facade contract; system prompt from `FACADE_AGENT_INSTRUCTIONS`. - **Transport:** `experimental_createMCPClient` + stdio, spawning the workspace `stagehand-facade` bin with an env allowlist (`STAGEHAND_*`/`BROWSERBASE_*` only); client closed in `finally`. - **Run:** `packages/integrations/vercel-ai/README.md`. Verified: contract test in CI (three tools discovered, no browser on connect); Browserbase smoke passed (heading cited with snapshot ID, clean child exit). Stacked on #2665. Co-authored-by: Shrey Pandya <shrey@browserbase.com>
Eve example for the Stagehand facade tool surface, using Eve-native authored tools (`defineTool`) instead of an MCP connection. - **Tools:** `run` / `snapshot` / `screenshot` — the shared facade contract; descriptions, schemas, and system prompt imported from `@browserbasehq/stagehand-integrations/facade`, never restated. - **Transport:** none — tools run in-process in the Eve world. Browser session is a module-level singleton with keep-alive + reconnect-by-session-id (`browser.sessionId`). - **Run:** `packages/integrations/eve/README.md`. `pnpm dev` / `pnpm start` route through a wrapper that forwards the SDK extension asset paths (needs #2676's env overrides, merged into this branch). Verified: contract test in CI; Browserbase smoke passed (agent cited snapshot ID `[0-19]`, heading "Example Domain"); local `eve dev` browser flow verified end-to-end. Stacked on #2665. --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> Co-authored-by: Shrey Pandya <shrey@browserbase.com>
Summary
Adds a Playwright-facade tool surface to
@browserbasehq/stagehand-integrations:run/snapshot/screenshottool contract (descriptions, input schemas, error strings) is string-pinned by unit tests so any drift from the reference surface fails CI.StagehandFacadeTools— tool implementation with a serialized call queue (one facade call at a time per instance) and per-page snapshot state.stagehand-facadestdio MCP bin — exposes the three tools over MCP stdio, returns screenshots as image content blocks, and redacts credentials from error output.Deliberate exclusions
contract.jsoncodegen + Python re-pin follow in a later PR.Gates
Smoke
Browserbase facade smoke (stdio bin + eve example)
Summary by cubic
Adds a Playwright-compatible Stagehand facade to
@browserbasehq/stagehand-integrations, exposingrun,snapshot, andscreenshotover MCP stdio and as a library export. Moves the package topackages/integrations/coreto fixeve devsnapshot EINVAL and align CI/test paths.New Features
stagehand-facadeMCP stdio server forrun,snapshot,screenshot; screenshots return as image blocks.StagehandFacadeToolswith a serialized call queue, per-page snapshot state, and a Playwright-compatible runtime.runwire schema drops top-leveloneOf; runtime validation viazod/v4.STAGEHAND_BROWSER,BROWSERBASE_API_KEY,BROWSERBASE_PROJECT_ID); new binstagehand-facade; exports./facadeand./facade/stdio-server.google/gemini-3.6-flash; update the facade model default; exportFACADE_AGENT_INSTRUCTIONS.Bug Fixes
waitForOutputfast on early host exit, treat invalid browser/model asStagehandFacadeConfigError, and pass only integral JPEG quality.packages/integrations/coreand update cache/vitest roots; regenerate Go embedded extension; increase docs SDK reference test timeout to 30s.Written for commit dbb0a7b. Summary will update on new commits.