fix(dom): include shadow-DOM form elements when offsetWidth/Height === 0#39
Draft
caffeinum wants to merge 2 commits into
Draft
fix(dom): include shadow-DOM form elements when offsetWidth/Height === 0#39caffeinum wants to merge 2 commits into
caffeinum wants to merge 2 commits into
Conversation
`isElementVisible` in the in-page DOM walker hard-fails when `offsetWidth === 0 || offsetHeight === 0`. Shadow-DOM-hosted form elements (e.g. Stripe Elements, password manager extensions, auth0 login widgets, custom input components) commonly report 0 dimensions on the inner control even when they're real interactive elements visible on screen — visibility is governed by the shadow host's layout, not the element's own offset box. The agent then can't see them in the element index and can't interact with them. Add a SHADOW_DOM_FORM_TAGS whitelist (input, select, textarea, button, a) and an isInsideShadowDom() helper. In buildDomTree, if an element is a form tag inside a shadow root, treat it as visible+top so isInteractiveElement runs and it lands in selector_map regardless of offsetWidth/Height. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…covered
Adds a regression test for the SHADOW_DOM_FORM_TAGS / isInsideShadowDom
exception in src/dom/dom_tree/index.js. The test:
1. Navigates (via data: URL — about:blank short-circuits selector_map)
to a page hosting a shadow root that contains an <input> and a
<button>, both styled width:0; height:0; display:block so they
report offsetWidth/Height === 0 while remaining rendered.
2. Sanity-checks the offset dimensions to confirm the bug surface
(offsetWidth=0 with display !== none) is actually exercised.
3. Asserts the shadow-DOM <input> and <button> reach selector_map.
Verified failure-on-baseline by checking out upstream/main's version of
src/dom/dom_tree/index.js (which lacks the fix) and re-running the test:
the new "discovers shadow-DOM input with offsetWidth/Height = 0" case
fails with "expected 0 to be greater than 0", proving the test
genuinely exercises the bug. Restoring HEAD's version makes it pass.
Also adds a guard test ensuring the whitelist stays a whitelist — a
plain <div> with offsetWidth=0/Height=0 inside a shadow root must NOT
be promoted by the shadow-DOM exception.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
isElementVisibleinsrc/dom/dom_tree/index.jshard-fails whenoffsetWidth === 0 || offsetHeight === 0. Shadow-DOM-hosted form elements (commonly seen in web components like Stripe Elements, password manager extensions, custom input widgets) frequently reportoffsetWidth/Height = 0even when they're real interactive elements visible on screen. The agent then can't see them in the element index and can't interact with them.Fix
Add a
SHADOW_DOM_FORM_TAGSwhitelist (input, select, textarea, button) and anisInsideShadowDom()helper. InbuildDomTree, if an element is a form tag inside a shadow root, treat it as visible regardless of offsetWidth/Height — its actual visibility is governed by the shadow host's layout, not the element's own offset box.Reference: Python upstream
Python
browser-useno longer uses an in-pagebuildDomTree.js; visibility is computed server-side via CDPDOMSnapshotinbrowser_use/dom/service.py::is_element_visible_according_to_all_parents(L242-336). It checksdisplay/visibility/opacity+snapshot_node.bounds, NOToffsetWidth/Height. Shadow roots are walked separately (L866-882) and pass through the same visibility func. Notably, the python file has a debug log at L897-912 explicitly flagging form elements inside iframes/shadow trees with missing bounds — so the failure mode is known upstream but unfixed there too (different code path, separate problem).So this is not a port of an existing python fix; it's a TS-side fix for an analogous gap. Framing it as parity would be inaccurate — framing it as a fresh bug fix for shadow-DOM form discovery is correct.
Test plan
<input>/<select>appear in the element index when their offsetWidth/Height = 0🤖 Generated with Claude Code