Skip to content

fix: inspect windows whose title changes during the read - #26

Merged
iamnbutler merged 1 commit into
mainfrom
title-churn
Oct 15, 2026
Merged

iamnbutler merged 1 commit into
mainfrom
title-churn

Conversation

@iamnbutler

@iamnbutler iamnbutler commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

In the Ace desktop tools study (githubnext/ace#427, D7 and R1), desktop_inspect in Accessibility mode on a Google Chrome window whose page title changes every 2 s always failed with "The desktop observation provider returned inconsistent response evidence: window-context window title." It failed twice in a row on Canary with 4618b33. Pixels mode worked. Any web app with a live title (unread counts, timers, chat) could not be inspected for input. Tracks #6.

Cause. The observation resolver takes the target window's title from WindowServer. The detached Accessibility traversal then reads kAXTitle again from the AX window it selects by exact window ID, and puts that title in the elements' window context. PeekabooBridgeDesktopObservationBinding.validateWindowEvidence required every window context's title to equal the resolved target's title, so a title that changed between the two reads failed the whole observation. Pixels mode has no traversal, so it passed.

Fix. A new last patch, peekaboo-title.patch, compares a window context's title only when the context names no window. A context with a window ID is already pinned to the target:

  • the same check requires its window ID to equal the target's (and the request's exact ID);
  • its WindowServer receipt must name that window ID, the target's owner PID and the target's bounds, and its bounds must equal the target's;
  • every process generation in the result must agree;
  • the traversal selects its AX window by the unique _AXUIElementGetWindow match for that ID and checks the process generation before and after.

A title adds nothing to that. A context without a window ID still has to match the title, and the capture's title check is unchanged (the capture reuses the resolved title). The resolved target keeps its title. The client's window_title now reports the traversal's newer title when there is one.

Input actions. None of them compare the window title. Snapshot-based clicks, inserts and keys bind by window ID, the WindowMutationIdentity receipt (PID, generation, bounds) and focused-element identity. AutomationElementResolver uses the title only when the context has no window ID. The focused-element receipt compares the element's own title, not the window's. So nothing else needed changing.

Verification

  • node dist/build.js --out … --scratch … builds with the full stack, including the client change.
  • With the full stack, PeekabooCore swift test:
    • new suite PeekabooBridgeObservationTitleChurnTests (stub services): a PID + window-ID Accessibility observation whose traversal title changed passes live (handleAuthorized) and offline (signed receipt) validation and returns both titles. A changed title with no window ID is still rejected (window-context window title), and so is a different window ID. Both tests fail without the fix.
    • PeekabooBridgeTests (115 tests) and every PeekabooTests.PeekabooBridge*Observation* suite (65 tests) pass.
  • npm run check (dist matches tsc).
  • No live desktop operations, because another tester owns the screen. Inspecting a live-title Chrome window still needs checking.

desktop_inspect in Accessibility mode on a Chrome window whose page title
changes every 2 s always failed with "The desktop observation provider
returned inconsistent response evidence: window-context window title."
(githubnext/ace#427, D7 and R1). The observation resolver takes the
window's title from WindowServer, and the detached Accessibility traversal
reads kAXTitle again later; the Bridge binding validator required the two
to be equal, so any live title (unread counts, timers, chat) made the
window uninspectable for input. Pixels mode has no traversal and passed.

A new last patch, peekaboo-title.patch, compares a window context's title
only when the context names no window. A context with a window ID is
already pinned to the target by that exact ID, its WindowServer receipt
(owner PID, process generation, bounds) and bounds, and the traversal
selects its AX window by the same ID. The resolved target keeps its title,
and the client now reports the traversal's newer title. Input actions bind
by window ID, receipt and focused-element identity, never the window
title, so they need no change.

Why: native/patches/peekaboo-title.patch | A window context's title is skipped only when its window ID matches the target; a context without a window ID must still match the title, and window ID, receipt, bounds and process generation stay exact.
@iamnbutler
iamnbutler merged commit 73c0513 into main Oct 15, 2026
2 checks passed
iamnbutler added a commit that referenced this pull request Oct 17, 2026
Accessibility inspection of a Chrome page whose title changes every 50 ms now succeeds (fixed by #26).

Closes #6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant