Skip to content

feat(evaluate): allow passing functions as evaluate arguments#41801

Open
pavelfeldman wants to merge 1 commit into
microsoft:mainfrom
pavelfeldman:evaluate-callbacks
Open

feat(evaluate): allow passing functions as evaluate arguments#41801
pavelfeldman wants to merge 1 commit into
microsoft:mainfrom
pavelfeldman:evaluate-callbacks

Conversation

@pavelfeldman

@pavelfeldman pavelfeldman commented Jul 15, 2026

Copy link
Copy Markdown
Member

Summary

  • When evaluate is called with the exposeFunctions option, functions passed as (part of) the argument are exposed as page bindings with unguessable names and materialize as functions in the page; calling one returns a promise of the client-side result.
  • Bindings are erased upon main frame navigation, scoping the callbacks to the execution context they were passed to.
  • Without the option, passing a function throws as before.
  • Works for page, frame, jsHandle and locator evaluate.
  • Component testing now passes callback props through this mechanism, replacing the ordinal-based __ctDispatchFunction dispatch.

Comment thread packages/playwright-core/src/client/jsHandle.ts
Comment thread packages/playwright-core/src/client/jsHandle.ts
Comment thread docs/src/api/params.md Outdated
Comment thread packages/playwright-core/src/client/jsHandle.ts
Comment thread packages/playwright-core/src/client/page.ts Outdated
Comment thread packages/playwright-core/src/client/page.ts
Comment thread packages/playwright-core/src/client/page.ts
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot and I took a first look at the failing CI.

🔴 The tracing.spec.ts failure looks caused by this PR

library/tracing.spec.ts:780 › should not emit after w/o before fails deterministically on all three browsers (chromium incl. arm/node22/node24, firefox, webkit). It asserts the exact before/after call events recorded in a trace around page.evaluate, and this PR rewrites how evaluate serializes and dispatches its arguments and result — so the recorded call sequence shifts. That's a real, PR-caused failure, not a flake.

Details

Caused by this PR

  • [chromium|firefox|webkit-library] › library/tracing.spec.ts:780 › should not emit after w/o before — the assertion at line 818 fails on every browser and both retries. The expected trace1 events are before (evaluate) → before "Wait for event console"after, but the run records the "Wait for event console" title on the first before and shifts the others (Expected -1 / Received +3). The test exercises page.evaluate / page.evaluateHandle, exactly the code paths this PR reworks (client/channelOwner.ts, client/frame.ts, client/jsHandle.ts, client/page.ts, plus the serializers). A consistent, all-browser, all-retry failure on an evaluate-tracing test lines up with this diff rather than with infra noise. Worth checking whether the new evaluate dispatch emits/orders its trace call events differently, and either restoring the prior order or updating the expectation if the new order is intended.

Pre-existing flake / infra

  • The 2 flaky entries — library/video.spec.ts:736 screencast › should work with video+trace and :699 › should capture full viewport on hidpi — passed on retry and are unrelated screencast flakes; nothing to triage there.

Triaged by the Playwright bot - agent run

When evaluate is called with the exposeFunctions option, functions
passed as (part of) the argument are exposed as page bindings with
unguessable names and materialize as functions in the page. The
page-side function calls the binding and returns a promise of the
result. The bindings are erased upon main frame navigation, scoping
the callbacks to the execution context they were passed to.

Without the option, passing a function throws as before.

Works for page, frame, jsHandle and locator evaluate. Component
testing now mounts components with callback props through this
mechanism instead of the ordinal-based __ctDispatchFunction dispatch.
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

1 failed
❌ [chromium] › mcp/annotate.spec.ts:57 › should capture multiple screenshots in one annotation @mcp-macos-latest-chromium

7759 passed, 1249 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

1 failed
❌ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace @frozen-time-library-chromium-linux

7 flaky ⚠️ [chromium-library] › library/chromium/chromium.spec.ts:177 › serviceWorker(), and fromServiceWorker() work `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/video.spec.ts:699 › screencast › should capture full viewport on hidpi `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/video.spec.ts:294 › screencast › should capture navigation `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/video.spec.ts:664 › screencast › should capture full viewport `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/video.spec.ts:699 › screencast › should capture full viewport on hidpi `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:434 › should produce network events, routing, and annotations for Service Worker (advanced) `@chromium-ubuntu-22.04-node22`
⚠️ [playwright-test] › ui-mode-trace.spec.ts:388 › should reveal errors in the sourcetab `@windows-latest-node22`

49877 passed, 1156 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot and I took a first look at the failing CI.

🔴 The tracing.spec.ts failure looks caused by this PR

library/tracing.spec.ts:780 › should not emit after w/o before fails deterministically on all three browsers, and the aggregated CI results show this test failing on no SHA other than this PR's — so it's a real, PR-caused failure, not a flake. The one MCP failure is a known unrelated flake.

Details

This PR reworks how evaluate serializes and dispatches its arguments and result (client/channelOwner.ts, client/frame.ts, client/jsHandle.ts, client/page.ts, plus the serializers), and the tracing test that broke asserts the exact before/after call events recorded around page.evaluate. That's exactly the code path the diff touches.

Caused by this PR

  • [chromium|firefox|webkit-library] › library/tracing.spec.ts:780 › should not emit after w/o before — fails on every browser (chromium incl. arm / node22 / node24, firefox, webkit) and on both retries. In the aggregated results DB this test is otherwise green everywhere — chromium 6 failed / 1902 runs, firefox 1/387, webkit 1/380 — and every one of those failures is on PR feat(evaluate): allow passing functions as evaluate arguments #41801 (sha 39101ae3ed); no other SHA or PR has ever failed it. A consistent, all-browser, all-retry failure that appears only on this PR points at the diff, not infra. The test exercises page.evaluate / page.evaluateHandle and checks the recorded beforebefore "Wait for event console"after sequence, so it's worth checking whether the new evaluate dispatch emits/orders its trace call events differently, and either restoring the prior order or updating the expectation if the new order is intended.

Pre-existing flake / infra

Triaged by the Playwright bot - agent run

@github-actions

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot and I took a look at the failing CI.

🟢 The PR is clear — the two remaining failures are known flakes

On the newest tested commit (ca1e005be7) the only two failures are video.spec.ts:736 and mcp/annotate.spec.ts:57, both pre-existing flakes that fail on main and unrelated PRs. The earlier all-browser tracing.spec.ts:780 failure was real and PR-caused, but it's already fixed on this commit.

Details

Fixed since the earlier red verdict

  • [chromium|firefox|webkit-library] › library/tracing.spec.ts:780 › should not emit after w/o before — failed on all three browsers on the previous commit 39101ae3ed (8 failures) and was genuinely caused by this PR's evaluate rework. On the newer commit ca1e005be7 it passes on every browser, and in the aggregated results DB it has never failed on any SHA other than 39101ae3ed. Nothing left to do here.

Pre-existing flake / infra

Triaged by the Playwright bot.

Triaged by the Playwright bot - agent run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants