Skip to content

fix: name the unmatched menu title and its siblings - #15

Merged
iamnbutler merged 1 commit into
mainfrom
menu-refusal
Oct 15, 2026
Merged

iamnbutler merged 1 commit into
mainfrom
menu-refusal

Conversation

@iamnbutler

Copy link
Copy Markdown
Contributor

In the Ace desktop tools study (githubnext/ace#427, scenario D1), desktop_menu with ["File", "Save"] in TextEdit was refused with "The literal menu path is unavailable without opening ancestor menus." The item is "Save…", so the title just didn't match; the message blamed lazy menus instead. The agent recovered only by listing menus.

resolve() in peekaboo-menu.patch now refuses an unmatched title with the component, its level, and the sibling titles it already read (in order, at most 30 and 300 characters, empty titles skipped). When exactly one sibling is the requested title plus a trailing "…" or "...", it names it:

No menu item titled "Save" under "File" (level 2 of 2); did you mean "Save…"? Titles must match exactly. Titles under "File": "New", "Open…", "Open Recent", "Close", "Save…", "Duplicate", ...

The ambiguous refusal keeps its wording and adds the component: The literal menu path is ambiguous: 2 items titled "Close" under "File" (level 2 of 2).

Matching is still exact UTF-16 and never accepts the suggestion. Budgets, roles, uniqueness and the invalidRequest pre-dispatch refusal are unchanged. The patches README now says titles match exactly, including ellipses.

Verification

  • Regenerated the patch from a checkout with the preceding stack applied; the four later patches still apply.
  • node dist/build.js --out … --scratch … with a fresh scratch builds.
  • With the full stack: PeekabooAutomationKit swift test --filter Menu (35 XCTest, 71 swift-testing) and PeekabooCore swift test --filter PeekabooBridgeTests (55 + 115) all pass. No test covers the AX menu-command traversal, so the message text was checked with a throwaway script over the new function.
  • No live menu calls: another agent was using the screen.

A menu command whose title had no literal match refused with "The literal
menu path is unavailable without opening ancestor menus", which blamed lazy
menus when the title simply differed (githubnext/ace#427: "Save" for
TextEdit's "Save…"). The refusal now names the component, its level, and up
to 30 sibling titles (300 characters) already read, and suggests a lone
sibling that only adds a trailing ellipsis. The ambiguous refusal also names
the component. Matching, budgets, roles and refusal kind are unchanged.

Why: native/patches/peekaboo-menu.patch | Menu titles match exact UTF-16; the refusal may suggest a near title but resolution must never accept one.
@iamnbutler
iamnbutler merged commit 034a695 into main Oct 15, 2026
2 checks passed
iamnbutler added a commit to githubnext/ace that referenced this pull request Oct 15, 2026
)

* fix(channel): pin desktop-tools with clearer menu and key refusals

Moves desktop-tools from 20cb1b5 to 034a695:
- githubnext/desktop-tools#15: a menu path with no match names the missing title, its level and the titles at that level, and suggests "Save…" for "Save".
- githubnext/desktop-tools#14: a key with no focused receiver says why, and points to the menu.

desktop_menu's description now says titles match exactly, ellipses included. The docs point to Settings → Computer use for permissions.

Found in the desktop tools study, D1 and D2 (#427).

* chore: confirm package.json notes after the desktop-tools pin bump
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant