Skip to content

use locator inputs for observe & extract - #2654

Merged
seanmcguire12 merged 32 commits into
v4-spikefrom
locator-inputs
Aug 10, 2026
Merged

use locator inputs for observe & extract#2654
seanmcguire12 merged 32 commits into
v4-spikefrom
locator-inputs

Conversation

@seanmcguire12

@seanmcguire12 seanmcguire12 commented Aug 9, 2026

Copy link
Copy Markdown
Member

why

  • observe & extract previously accepted selector strings instead of locator's. for consistency across our APIs, this PR changes them to accept locator's

what changed

  • replaced selector / ignoreSelectors observe and extract protocol options with locator descriptor fields
  • migrated extension snapshot, observe, and extract internals to use locator descriptors, including nth
  • bypasses the current server-side cache for locator-scoped observe/extract calls to avoid sending locator descriptors through the selector-shaped cache contract
  • updated TypeScript, Python, and Go SDK public APIs to accept page-bound locator wrapper objects instead of protocol shaped locator schemas
  • added validation for checking that supplied locators belong to the resolved target page before sending RPC requests
  • serializes observe/extract locator options in TS batch callbacks after resolving the operation page
  • updates v4 docs and examples to show page.locator(...) / page.Locator(...) scoping
  • also updates evals to use locator-based extract scoping

returned selectors remain plain output data. Action.selector and snapshot selector maps are not wrapped as live locators by this PR

SDK shapes

TypeScript:

await stagehand.extract("Extract text", Schema, {
  page,
  locator: page.locator("main"),
  ignoreLocators: [page.locator("nav"), page.locator(".ad").nth(2)],
});

Python:

await stagehand.extract(
    "Extract text",
    Schema,
    page=page,
    locator=page.locator("main"),
    ignore_locators=[page.locator("nav")],
)

Go:

result, err := stagehand.Extract[resultType](
	ctx,
	client,
	"Extract text",
	&stagehand.StagehandClientExtractOptions{
		Page:    page,
		Locator: page.Locator("main"),
		IgnoreLocators: []*stagehand.PageLocator{
			page.Locator("nav"),
		},
	},
)

behaviour notes:

  • page remains optional. If omitted, Stagehand resolves the active page and validates locators against that page
  • Locator descriptors include selector and optional nth internally
  • Locator-scoped observe/extract calls bypass the current server-side cache; unscoped calls keep existing cache behavior
  • Ignored locators exclude matched nodes and descendants from the snapshot context
  • observe results and act(Action) inputs still use selector strings as action data

test plan

  • added protocol tests to cover the new observe/extract wire shape for locator & ignoreLocators
  • extension tests cover locator-scoped observe/extract snapshots, ignored locator subtrees, cache bypass behavior, & nth handling
  • TS SDK tests cover public Locator wrapper inputs, descriptor serialization, & cross-page validation
  • Python SDK tests cover public Locator wrapper inputs, descriptor serialization, and cross-page validation
  • Go SDK tests cover handwritten observe/extract client options, *PageLocator conversion, cross-page validation, and generated model compatibility for the SDK/internal packages
  • also updated docs tests

Summary by cubic

Switch observe, extract, and act to accept page-bound locator inputs instead of selector strings, standardizing scoping, enabling precise nth targeting, and letting you exclude subtrees. Locator-scoped calls bypass server caching; SDKs validate that locators belong to the target page.

  • New Features

    • Added locator and ignoreLocators to act/observe/extract; all support .nth for disambiguation. text= locators are not yet supported for snapshot scoping.
    • TypeScript, Python, and Go SDKs accept page-bound Locator objects for all three methods and enforce same-page validation; TS batch callbacks serialize locators after resolving the operation page.
    • Extension snapshotting and a11y scoping use locator descriptors (focusLocator/ignoreLocators) and honor .nth; added targeted tests.
    • Caching: locator-scoped act/observe/extract bypass server cache reads/writes; cache inputs omit locator descriptors; responses show metadata.cache.status as DISABLED.
    • Protocol schemas: ActOptions adds ignore_locators; observe/extract now use locator/ignore_locators.
    • Docs and evals updated to use page.locator("...") and xpath=...; evals Vitest config updated. Go SDK: flattened per-call fields for Act options and fixed nil IgnoreLocators handling.
  • Migration

    • Replace selector with locator: page.locator("...") and ignoreSelectors with ignoreLocators: [page.locator("..."), ...] in observe/extract.
    • Use .nth(index) to target a specific match; include xpath= when using XPath locators.
    • Go SDK: move per-call fields to top-level of StagehandClientActOptions (and already for Observe/Extract), e.g., use Cache, Timeout, Model, Locator, IgnoreLocators, Variables directly.
    • Returned selectors in results remain strings; no change to action inputs/outputs.

Written for commit 85138cd. Summary will update on new commits.

Review in cubic

@changeset-bot

changeset-bot Bot commented Aug 9, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 6f21169

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@seanmcguire12
seanmcguire12 marked this pull request as ready for review August 9, 2026 21:29
@seanmcguire12
seanmcguire12 requested a review from a team as a code owner August 9, 2026 21:29
@seanmcguire12
seanmcguire12 removed the request for review from a team August 9, 2026 21:29

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 47 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/sdk-go/extract.go
Comment thread packages/extension/tests/cache-service.test.ts Outdated
Comment thread packages/docs/v4/basics/observe.mdx
Comment thread packages/sdk-go/stagehand.go
Comment thread packages/docs/v4/reference/stagehand.mdx Outdated
Comment thread packages/evals/tasks/bench/extract/extract_geniusee_2.ts
Comment thread packages/docs/v4/basics/observe.mdx
Comment thread packages/extension/callbackBatch.ts
Comment thread packages/extension/understudy/a11y/snapshot/capture.test.ts
Comment thread packages/sdk-ts/src/clientLocatorOptions.ts

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/extension/services/observeService.ts Outdated
Comment thread packages/extension/services/extractService.ts Outdated
Comment thread packages/extension/services/observeService.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 52 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/docs/v4/basics/observe.mdx
Comment thread packages/extension/tests/cache-service.test.ts
Comment thread packages/docs/v4/reference/stagehand.mdx
Comment thread packages/docs/v4/reference/stagehand.mdx
Comment thread packages/extension/services/cacheService.ts
Comment thread packages/sdk-go/behavior_regression_test.go
Comment thread packages/sdk-go/stagehand.go

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread packages/docs/v4/reference/stagehand.mdx
Comment thread packages/docs/v4/reference/stagehand.mdx
seanmcguire12 and others added 3 commits August 9, 2026 19:30
# why
- for consistency across `act`/`extract`/`observe`
# what changed
this PR extends locator-based scoping to
instruction-based/nondeterministic `act`
- adds `ignoreLocators` to the `act` protocol options
- passes `locator` and `ignoreLocators` through act snapshot capture so
the model only sees the intended page context
- includes act locator descriptors in cache keys
- updates TS, Python, & Go SDK public APIs to accept page-bound locator
wrapper objects for `act`
- validates that supplied act locators belong to the resolved target
page before sending RPC requests

## SDK Shape

TypeScript:

```ts
await stagehand.act("Click the checkout button", {
  page,
  locator: page.locator("main"),
  ignoreLocators: [page.locator("nav"), page.locator(".promo-modal")],
});
```

Python:

```py
await stagehand.act(
    "Click the checkout button",
    page=page,
    locator=page.locator("main"),
    ignore_locators=[page.locator("nav"), page.locator(".promo-modal")],
)
```

Go:

```go
_, err := client.Act(ctx, stagehand.ActInstruction("Click the checkout button"), &stagehand.StagehandClientActOptions{
	Page:    page,
	Locator: page.Locator("main"),
	IgnoreLocators: []*stagehand.PageLocator{
		page.Locator("nav"),
		page.Locator(".promo-modal"),
	},
})
```
## behavioural notes:
- observed actions still execute against their `Action.selector` data
directly. Locator scoping only affects
instruction-based/nondeterministic `act`
- `page` remains optional. If omitted, Stagehand resolves the active
page & validates locators against that page
- locator descriptors include `selector` & optional `nth` internally
- `ignoreLocators` removes matched nodes & descendants from the act
planning snapshot
- act cache data now includes `locator` and `ignoreLocators`, so scoped
planning does not reuse incompatible cached actions
- TS batch callbacks now extend the existing locator option
serialization path to `act`
- Go `StagehandClientActOptions` now uses explicit client fields instead
of embedding generated protocol `ActOptions`


# test plan
- added protocol tests to cover the new act wire shape for `locator` and
`ignoreLocators`
- extension act tests cover focused locator scope, ignored locator
subtrees, and locator-filtered planning prompts
- extension cache tests cover act, observe, and extract locator
descriptors in cache data
- TS SDK tests cover act locator serialization, batch callback locator
serialization, and cross-page validation
- Python SDK tests cover public act `Locator` inputs, descriptor
serialization, and cross-page validation
- Go SDK tests cover act `*PageLocator` conversion, cross-page
validation, and generated model compatibility for the SDK/internal
packages
- docs tests cover the updated v4 act reference and examples


<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds locator scoping to instruction-based `act()` with a new
`ignoreLocators` option so the model plans within a focused DOM and
excludes ignored regions. Scoped `act()` bypasses server-side caching
and sends locators to snapshot capture for both planning and
verification.

- **New Features**
- Add `ignoreLocators` to `act` options; pass `locator` and
`ignoreLocators` to snapshot capture for planning and the verification
step.
- Locator-scoped instruction-based `act()` bypasses server-side result
caching and reports `metadata.cache.status: DISABLED`.
  - Protocol: add `ignore_locators` to `ActOptions` in the v4 schema.
- Locator scoping applies to instruction-based `act()` only; replays of
recorded `Action`s use their own selectors.

- **SDKs**
- `sdk-ts`: `act` accepts locator wrappers and serializes them in direct
calls and callback batches (page resolution + locator serialization for
`act`, `observe`, `extract`); cross-page validation added.
- `sdk-python`: `act` accepts `locator` and `ignore_locators` as
page-bound locators with cross-page validation.
- `sdk-go`: `StagehandClientActOptions` uses flattened fields (`Page`,
`Locator`, `IgnoreLocators`, `Timeout`, `Variables`, `Model`, `Cache`);
`act` serializes `PageLocator`s with cross-page validation.
- Docs updated with `act()` scoping examples, cache bypass behavior, and
Go option shape; speed and caching guides show scoped `act()` usage.

<sup>Written for commit 6e038da.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2655?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->
@seanmcguire12
seanmcguire12 merged commit 168996c into v4-spike Aug 10, 2026
50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants