Skip to content

feat(integrations): Facade tool surface (run/snapshot/screenshot) - #2665

Merged
shrey150 merged 14 commits into
mainfrom
miguel/facade-core
Aug 10, 2026
Merged

feat(integrations): Facade tool surface (run/snapshot/screenshot)#2665
shrey150 merged 14 commits into
mainfrom
miguel/facade-core

Conversation

@miguelg719

@miguelg719 miguelg719 commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a Playwright-facade tool surface to @browserbasehq/stagehand-integrations:

  • Contract promoted verbatim from the facade reference — the run / snapshot / screenshot tool contract (descriptions, input schemas, error strings) is string-pinned by unit tests so any drift from the reference surface fails CI.
  • StagehandFacadeTools — tool implementation with a serialized call queue (one facade call at a time per instance) and per-page snapshot state.
  • stagehand-facade stdio MCP bin — exposes the three tools over MCP stdio, returns screenshots as image content blocks, and redacts credentials from error output.

Deliberate exclusions

  • No disable-code flag (per review decision).
  • contract.json codegen + Python re-pin follow in a later PR.

Gates

  • Build: green
  • Typecheck: green
  • Unit tests: 17/17 green

Smoke

Browserbase facade smoke (stdio bin + eve example)
SMOKE_RESULT_START
BROWSERBASE FACADE SMOKE — /tmp/wt-facade-eve (miguel/facade-eve-example, HEAD 81e2e8817)

SETUP
- pnpm install --frozen-lockfile: already up to date (160ms). Built @browserbasehq/stagehand-integrations (tsdown, 410ms); sdk-ts dist already present.
- Credentials: no BROWSERBASE_* in process env. Main .env has 3 commented prod/dev key pairs; packages/integrations/deepagents/examples/managed/.env has one ACTIVE BROWSERBASE_API_KEY (no project id). All loaded programmatically, never printed. Model key: OPENAI_API_KEY active in main .env.

PART 2 — stdio bin direct (driver: /tmp/wt-facade-eve/facade-smoke.mjs, JSON-RPC over stdio, STAGEHAND_BROWSER=browserbase)
- The 3 commented .env key pairs are STALE: run(code) fails in ~150ms with "Failed to upload the Stagehand extension to Browserbase" (auth-level failure, so requests did reach Browserbase).
- PASS with the active managed-.env key (projectId omitted; facade treats it as optional):
  - initialize (125-141ms): serverInfo stagehand-facade@4.0.0
  - tools/list (~1ms): exactly [run, screenshot, snapshot]
  - run {code: goto example.com + return title} (2186ms): returned "Example Domain"
  - snapshot {} (176ms): formatted tree with bracketed IDs — `[0-6] RootWebArea: Example Domain`, `[0-19] heading: Example Domain`, `[0-24] link: Learn more`
  - run {actions:[{op:"click", id:"0-19"}]} using real snapshot id (225ms): `{"completed":1,"url":"https://example.com/"}`
  - screenshot {} (213ms): image content block, mimeType image/png, base64len 33896, PNG magic bytes verified
  - stdin EOF → clean exit code 0. Total 3.9s.

PART 3 — eve example (driver: /tmp/wt-facade-eve/eve-smoke.mjs; bridge → eve build → eve start → eve invoke "Open example.com and return its heading", model gpt-5-mini via OPENAI_API_KEY) — PARTIAL
- Bridge (tsx src/facade-bridge.ts) starts in ~350ms, prints URL/token; clean SIGTERM exit 0 every run.
- `eve dev`/`eve invoke` without a prebuilt server FAILS in this workspace: dev-runtime snapshot copy EINVAL — "cannot copy .../packages/integrations/examples to a subdirectory of self .../examples/eve/.eve/dev-runtime/snapshots/.../source/packages/integrations/examples". Workaround: `eve build` + `eve start` + `eve invoke -u`.
- Gotcha: the connection URL/token from agent/connections/stagehand.ts are evaluated at BUILD time — must run `eve build` with STAGEHAND_FACADE_MCP_URL/TOKEN exported (README implies runtime-only).
- With plain prompt, invoke exits 0 in ~5.7s with "Example Domain" — but a counting proxy in front of the bridge proved the agent only did initialize+tools/list, answering from model memory, not the browser.
- With a verify-with-tools prompt (2 runs, reproducible): stagehand__snapshot succeeds end-to-end through eve → HTTP bridge → stdio facade → Browserbase (live snapshot of blank page returned). But every stagehand__run attempt fails EVE-SIDE with AI_InvalidToolInputError/AI_TypeValidationError before reaching the bridge (proxy saw zero tools/call:run) — likely eve's AI-SDK input validation choking on RUN_INPUT_SCHEMA's top-level `oneOf` in /tmp/wt-facade-eve/packages/integrations/src/facade/contract.ts (the same {code} payload validates fine when sent to the stdio server directly). Agent then returned a completed-status message asking how to proceed; heading never fetched via browser.

VERDICT: facade stdio surface over Browserbase fully PASSES. Eve example: bridge/connection/snapshot path works; `run` tool unusable from eve (schema-validation bug, reproducible 2/2) plus two doc/workspace issues (dev-runtime self-copy EINVAL, build-time env requirement).
SMOKE_RESULT_END

Summary by cubic

Adds a Playwright-compatible Stagehand facade to @browserbasehq/stagehand-integrations, exposing run, snapshot, and screenshot over MCP stdio and as a library export. Moves the package to packages/integrations/core to fix eve dev snapshot EINVAL and align CI/test paths.

  • New Features

    • stagehand-facade MCP stdio server for run, snapshot, screenshot; screenshots return as image blocks.
    • StagehandFacadeTools with a serialized call queue, per-page snapshot state, and a Playwright-compatible runtime.
    • Contract pinned by tests; run wire schema drops top-level oneOf; runtime validation via zod/v4.
    • Env-based config for local or Browserbase (STAGEHAND_BROWSER, BROWSERBASE_API_KEY, BROWSERBASE_PROJECT_ID); new bin stagehand-facade; exports ./facade and ./facade/stdio-server.
    • Protocol/models: add google/gemini-3.6-flash; update the facade model default; export FACADE_AGENT_INSTRUCTIONS.
  • Bug Fixes

    • Default local facade browser to headed mode.
    • Broader credential redaction (Browserbase, Google, bearer tokens).
    • Stability: bound shutdown during in-flight launch (5s), fail waitForOutput fast on early host exit, treat invalid browser/model as StagehandFacadeConfigError, and pass only integral JPEG quality.
    • CI/tests: move to packages/integrations/core and update cache/vitest roots; regenerate Go embedded extension; increase docs SDK reference test timeout to 30s.

Written for commit dbb0a7b. Summary will update on new commits.

Review in cubic

@changeset-bot

changeset-bot Bot commented Aug 9, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: dbb0a7b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 10 files

Architecture diagram
sequenceDiagram
    participant Client as MCP Client
    participant Server as stagehand-facade stdio bin
    participant Config as ConfigFromEnv
    participant Tools as StagehandFacadeTools
    participant Stagehand as Stagehand
    participant Runtime as PlaywrightCompat Runtime
    participant Browser as Browserbase/Local

    Note over Client,Browser: NEW: Playwright-facade tool surface over MCP stdio

    Client->>Server: initialize
    Server-->>Client: serverInfo stagehand-facade@4.0.0
    Client->>Server: tools/list
    Note over Server: custom ListTools handler returns pinned FACADE_TOOLS contract verbatim (no browser launch required)
    Server-->>Client: run / snapshot / screenshot wire JSON schemas

    Note over Client,Browser: First tools/call lazily creates resources (once per process)
    Client->>Server: tools/call run {code}
    Server->>Server: CodeModeRunInputSchema.parse (code XOR actions)
    Server->>Config: stagehandFacadeConfigFromEnv()
    Config-->>Server: STAGEHAND_BROWSER + BROWSERBASE_API_KEY/PROJECT_ID + model config
    alt STAGEHAND_BROWSER=browserbase (or API key present)
        Server->>Browser: browserbase.launch({apiKey, projectId?})
    else local (default headless)
        Server->>Browser: localBrowser.launch({headless: true})
    end
    Server->>Stagehand: Stagehand.create({browser, model...})
    Stagehand-->>Server: stagehand
    Server->>Tools: new StagehandFacadeTools(stagehand) — serialized queue starts
    Tools->>Tools: enqueue → runNow → activePage()

    Tools->>Stagehand: experimentalBatch(PRELUDE + user code + EPILOGUE, 60s timeout)
    Stagehand->>Runtime: createPlaywrightCompatRuntime(batchStagehand) serialized into page
    Runtime-->>Stagehand: compat page / context / browser proxies
    Stagehand-->>Tools: RunEnvelope { value | executionError }
    alt success
        Tools-->>Client: result value
    else user code threw
        Tools->>Tools: rethrow executionError as Error with original stack
    end
    Server->>Server: sanitizeErrorMessage (redacts sk-... / apiKey / token / signingKey)
    Server-->>Client: text result or isError text

    Note over Client,Browser: snapshot hydrates per-page id map
    Client->>Server: tools/call snapshot {includeIframes}
    Server->>Tools: SnapshotInputSchema.parse → enqueue → snapshotNow
    Tools->>Tools: page.snapshot() and store {url, xpathById} keyed by page.pageId
    Tools-->>Client: formatted tree with bracketed [a-b] ids

    Note over Client,Browser: run {actions} resolves ids against latest snapshot
    Client->>Server: tools/call run {actions:[{op:'click',id:'0-19'}]}
    Server->>Tools: RefActionSchema parse → enqueue → runActionsNow
    Tools->>Tools: activePage() + lookup snapshot state for that page
    alt no snapshot for this page
        Tools-->>Client: NO_HYDRATED_SNAPSHOT_ERROR
    else page.url() != snapshot.url
        Tools-->>Client: NAVIGATED_SNAPSHOT_ERROR
    else id missing from xpathById
        Tools-->>Client: staleSnapshotIdError(id)
    else all ids hydrate to xpath= selectors
        Tools->>Stagehand: experimentalBatch(ACTION_RUNNER_SOURCE, hydrated actions)
        Stagehand-->>Tools: {completed}
        Tools-->>Client: {completed, url}
    end

    Note over Client,Browser: screenshot returns image content block
    Client->>Server: tools/call screenshot {type, fullPage, quality}
    Server->>Tools: ScreenshotInputSchema.parse → enqueue → screenshotNow
    Tools->>Tools: page.screenshot() → base64 data + mimeType (png/jpeg)
    Server-->>Client: text 'Screenshot captured.' + image content block

    Note over Server,Browser: Shutdown on SIGINT/SIGTERM/stdin EOF closes in order
    Server->>Stagehand: stagehand.close()
    Server->>Browser: browser.close() → exit 0 (1 if unclean)
Loading

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/integrations/core/src/facade/stdio-server.ts
Comment thread packages/integrations/core/src/facade/contract.ts
Comment thread packages/integrations/src/facade/stdio-server.ts Outdated
Comment thread packages/integrations/core/src/facade/runtime.ts
Comment thread packages/integrations/core/src/facade/runtime.ts
Comment thread packages/integrations/core/src/facade/config.ts
Comment thread packages/integrations/core/src/facade/runtime.ts
Comment thread packages/integrations/core/src/facade/runtime.ts
Comment thread packages/integrations/core/tests/facade-server.test.ts
Comment thread packages/integrations/core/src/facade/tools.ts
@miguelg719

Copy link
Copy Markdown
Collaborator Author

Update: dropped the top-level oneOf from the run wire schema (b6b48aa4f).

The Browserbase smoke via the Eve example surfaced that AI-SDK-based MCP clients (Eve, Vercel AI SDK) reject tool input schemas with a top-level oneOf, failing every run call client-side before it reaches the server (reproducible 2/2; the same payload validated fine against the stdio server directly). The code/actions exclusivity remains in the tool description and is enforced at runtime by CodeModeRunInputSchema — the contract test now pins the oneOf absence and the three runtime-enforcement cases, with the deviation documented inline.

Re-smoke after the fix (Eve → HTTP bridge → stdio facade → Browserbase, model gpt-5-mini): proxy confirmed tools/call:run and tools/call:snapshot reached the facade; agent response: "I loaded https://example.com with Stagehand and verified the live snapshot. The page heading is: Example Domain. Evidence from the snapshot: "[0-19] heading: Example Domain"" — exit 0.

Follow-up (tracked for the deepagents consolidated pass on #2653): mirror the same wire-schema relaxation in the Python server so both hosts advertise identical bytes.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread packages/integrations/core/src/facade/tools.ts Outdated
Comment thread packages/integrations/core/src/facade/stdio-server.ts
Comment thread packages/integrations/core/src/facade/stdio-server.ts
Comment thread packages/integrations/core/src/facade/config.ts
@miguelg719

Copy link
Copy Markdown
Collaborator Author

Added commit 28c29f2 exposing the facade library surface via a ./facade package export (@browserbasehq/stagehand-integrations/facade). Both example PRs consume it: #2666 (Eve, native defineTool) imports the tool contract directly, and #2670 (Vercel AI SDK, MCP over stdio) wraps the same contract behind an MCP server.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread packages/integrations/core/src/facade/contract.ts
…screenshot) with stdio MCP host

Promotes the playwrightCompatRuntime and the run/snapshot/screenshot tool
contract from the facade experiment into packages/integrations as product
code: contract.ts carries the reference tool definitions verbatim (schemas,
descriptions, error strings), tools.ts adapts StagehandFacadeTools to a live
Stagehand handle with per-page snapshot state and a serialized call queue,
and a new stagehand-facade stdio bin exposes the three tools over MCP with
screenshots as image content blocks.
AI-SDK-based MCP clients (Eve, Vercel AI SDK) reject tool input schemas
with a top-level oneOf, failing every run call client-side before it
reaches the server. The code/actions exclusivity stays in the tool
description and is enforced at runtime by CodeModeRunInputSchema.
Native consumers (Eve defineTool, Vercel AI SDK tools) need direct access
to StagehandFacadeTools and the pinned contract, not just the stdio bin.
- widen credential redaction (Browserbase, Google, bearer tokens)
- bound shutdown against an in-flight launch (5s race)
- surface unsupported model names as StagehandFacadeConfigError
- only forward integral jpeg quality to page.screenshot
- fail waitForOutput fast when the host exits before ready
One system prompt shared by every host example instead of three
hand-authored variants.
…mples

packages/integrations/ becomes a grouping directory: core/ is the
@browserbasehq/stagehand-integrations package, and integration examples
(deepagents, eve, vercel-ai) sit beside it. Fixes eve dev's snapshot
EINVAL: eve copies a workspace dependency's package root, which
previously contained the eve example itself.
@miguelg719
miguelg719 requested a review from a team as a code owner August 10, 2026 07:12
@miguelg719
miguelg719 changed the base branch from v4-spike to main August 10, 2026 07:12
The protocol commits on this branch updated the schema without running
the Python generator; CI's generate.py --check caught the drift.
The Gemini 3.6 Flash protocol change on this branch made the Go-embedded
extension stale (extensionpack --check and the wheel-smoke build both
fail on it). The docs sdk-reference parity test times out at its 5s
default on cold PR runners (observed 5090ms) — give the multi-language
surface extraction a real budget.
@miguelg719 miguelg719 changed the title feat(integrations): Playwright-facade tool surface (run/snapshot/screenshot) feat(integrations): Facade tool surface (run/snapshot/screenshot) Aug 10, 2026
@mintlify

mintlify Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
stagehand 🟢 Ready View Preview Aug 10, 2026, 3:38 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2 files (changes from recent commits).

Confidence score: 2/5

  • In packages/integrations/core/src/facade/config.ts, the default local stdio facade now launches headed Chromium, which can fail on headless/container hosts without $DISPLAY, creating a concrete startup regression for non-interactive environments. Restore a headless default (or gate headed mode behind an explicit environment/control flag) to de-risk runtime failures.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/integrations/core/src/facade/config.ts">

<violation number="1" location="packages/integrations/core/src/facade/config.ts:82">
P1: The default local stdio facade no longer starts in headless/container hosts: it launches headed Chromium, which fails without `$DISPLAY`. Keep the non-interactive default headless (or add an explicit environment-controlled opt-in for headed debugging).</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

...(browserbaseProjectId ? { projectId: browserbaseProjectId } : {}),
},
}
: { type: "local", launchOptions: { headless: false } },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: The default local stdio facade no longer starts in headless/container hosts: it launches headed Chromium, which fails without $DISPLAY. Keep the non-interactive default headless (or add an explicit environment-controlled opt-in for headed debugging).

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/integrations/core/src/facade/config.ts, line 82:

<comment>The default local stdio facade no longer starts in headless/container hosts: it launches headed Chromium, which fails without `$DISPLAY`. Keep the non-interactive default headless (or add an explicit environment-controlled opt-in for headed debugging).</comment>

<file context>
@@ -79,7 +79,7 @@ export function stagehandFacadeConfigFromEnv(
             },
           }
-        : { type: "local", launchOptions: { headless: true } },
+        : { type: "local", launchOptions: { headless: false } },
     stagehand,
   };
</file context>

@shrey150
shrey150 merged commit b7a543a into main Aug 10, 2026
49 checks passed
shrey150 pushed a commit that referenced this pull request Aug 10, 2026
CrewAI example for the Stagehand facade tool surface, consuming it as a
standard MCP server over stdio.

- **Tools:** `run` / `snapshot` / `screenshot` — the shared facade
contract; agent backstory is the canonical facade instructions text,
pinned to the TS constant by a drift test (skips when lifted out of the
repo).
- **Transport:** `crewai-tools` `MCPServerAdapter` + stdio, spawning the
built `stagehand-facade` bin with an env allowlist
(`STAGEHAND_*`/`BROWSERBASE_*` only). Screenshots are saved to files
(CrewAI's adapter drops MCP image blocks — documented in the README).
- **Run:** `packages/integrations/crewai/README.md` (uv project, mirrors
the deepagents conventions). Tests run via `uv run pytest` locally and
in review; they are not wired into CI (no Python job covers this path).

Verified: contract + adapter + drift tests pass locally; Browserbase
smoke passed (heading cited with snapshot ID, clean child shutdown).

Stacked on #2665.
shrey150 pushed a commit that referenced this pull request Aug 10, 2026
Mastra example for the Stagehand facade tool surface, consuming it as a
standard MCP server over stdio.

- **Tools:** `run` / `snapshot` / `screenshot` — the shared facade
contract; system prompt from `FACADE_AGENT_INSTRUCTIONS`.
- **Transport:** Mastra `MCPClient` + stdio, spawning the workspace
`stagehand-facade` bin with an env allowlist
(`STAGEHAND_*`/`BROWSERBASE_*` only); `client.disconnect()` in
`finally`.
- **Run:** `packages/integrations/mastra/README.md`.

Verified: contract test in CI; Browserbase smoke passed ("Heading (from
snapshot [0-19]): Example Domain", no zombie processes).

Stacked on #2665.
shrey150 added a commit that referenced this pull request Aug 10, 2026
Vercel AI SDK example for the Stagehand facade tool surface, consuming
it as a standard MCP server over stdio.

- **Tools:** `run` / `snapshot` / `screenshot` — the shared facade
contract; system prompt from `FACADE_AGENT_INSTRUCTIONS`.
- **Transport:** `experimental_createMCPClient` + stdio, spawning the
workspace `stagehand-facade` bin with an env allowlist
(`STAGEHAND_*`/`BROWSERBASE_*` only); client closed in `finally`.
- **Run:** `packages/integrations/vercel-ai/README.md`.

Verified: contract test in CI (three tools discovered, no browser on
connect); Browserbase smoke passed (heading cited with snapshot ID,
clean child exit).

Stacked on #2665.

Co-authored-by: Shrey Pandya <shrey@browserbase.com>
shrey150 added a commit that referenced this pull request Aug 10, 2026
Eve example for the Stagehand facade tool surface, using Eve-native
authored tools (`defineTool`) instead of an MCP connection.

- **Tools:** `run` / `snapshot` / `screenshot` — the shared facade
contract; descriptions, schemas, and system prompt imported from
`@browserbasehq/stagehand-integrations/facade`, never restated.
- **Transport:** none — tools run in-process in the Eve world. Browser
session is a module-level singleton with keep-alive +
reconnect-by-session-id (`browser.sessionId`).
- **Run:** `packages/integrations/eve/README.md`. `pnpm dev` / `pnpm
start` route through a wrapper that forwards the SDK extension asset
paths (needs #2676's env overrides, merged into this branch).

Verified: contract test in CI; Browserbase smoke passed (agent cited
snapshot ID `[0-19]`, heading "Example Domain"); local `eve dev` browser
flow verified end-to-end.

Stacked on #2665.

---------

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Co-authored-by: Shrey Pandya <shrey@browserbase.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants