Skip to content

feat(agents-mobile): native slash-command composer for the Horton prompt#4533

Open
msfstef wants to merge 16 commits into
mainfrom
msfstef/richer-horton-text-input-mobile
Open

feat(agents-mobile): native slash-command composer for the Horton prompt#4533
msfstef wants to merge 16 commits into
mainfrom
msfstef/richer-horton-text-input-mobile

Conversation

@msfstef

@msfstef msfstef commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

What this is

This brings the mobile (Expo / React Native) Horton prompt up to feature/experience parity with the desktop ProseMirror composer: native slash-command autocomplete and structured composer_input payloads — on both the in-session and new-session inputs.

It's built as a native composer (a real RN TextInput), deliberately not a WebView embedding the desktop ProseMirror editor — prosemirror-view needs a real DOM and can't run under react-native-web, so the ProseMirror path would require a WebView, not merely allow one. The design, alternatives, and trade-offs are documented below.

🤖 Developed with Claude (hence the label). Manually verified on iOS + Android simulators; typecheck, lint, and the unit suites are green (45 mobile tests + the runtime composer-input suite).


Why native, not a WebView

The desktop composer is ProseMirror, which is DOM-only. We could have embedded it in a WebView / Expo DOM component, but the only reuse that actually matters — the composer_input wire contract and the slash-command grammar — is pure logic we can share losslessly. A WebView typing surface would import WKWebView caret/keyboard/IME friction and force an awkward native-popover-over-WebView bridge, all for editor code we don't need. So: native input, shared logic.


What's in here

1. Shared the slash grammar (no desktop behaviour change)
Lifted the serializer + slash-command grammar out of the desktop-only ComposerEditor into agents-runtime/src/composer-input.ts (exported via /client), so desktop and mobile derive nodes / highlighting / autocomplete from one source of truth. The desktop call sites just repoint their imports — behaviour-neutral for desktop — and the serializer tests moved into the runtime suite.

2. Native in-session composer
NativeComposer.tsx drives slash autocomplete on the existing SessionScreen TextInput: trigger detection from the shared grammar (no caret coordinates), a native keyboard-docked suggestion menu, and sending via createSendComposerInputAction. Discovery uses the built-in db.collections.slashCommands the screen already holds, falling back to the entity type's static slash_commands (static commands aren't materialised into the per-entity collection — the same fallback desktop uses).

3. New-session spawn parity + forward-compatible plumbing
The same autocomplete on NewSessionScreen, spawning with a composer_input initial message. No server change — the spawn endpoint already accepts a structured initialMessage + initialMessageType + an args record; we only widened the mobile spawnEntity wrapper and generalised its args channel so future schema-driven spawn args / model settings / attachments slot in additively rather than needing a re-plumb.

4. Inline command "badges"
Recognised /command tokens (and their declared argument words) are highlighted inline — the command in bold accent, the argument in a quieter accent, sharing a faint tint — mirroring the desktop badge's command + argument slot.

5. Platform fixes found while testing on device

  • iOS auto-grow: switching the input to child <Text> (needed for inline highlighting) breaks iOS's onContentSizeChange (RN #13732), so the input now sizes to content intrinsically.
  • Android keyboard: Android's default resize mode already lifts the bottom-anchored composer above the keyboard; the manual translateY double-offset it (sending it toward the top of the screen) — now applied on iOS only.

Key decisions & trade-offs

  • One shared grammar, pinned by a test. Mobile (regex over the source string) and desktop (walking the ProseMirror doc) can diverge at the edges — e.g. the grammar is intentionally lowercase-only — so a runtime test pins that boundary to stop the two silently drifting.
  • The popover is native and docked above the keyboard, not caret-anchored. RN TextInput exposes selection indices but not caret x/y (there's no coordsAtPos analog), and an above-keyboard list is the Slack/Discord/iMessage pattern anyway.
  • Badges are a flat tint + font weight, not a rounded chip. The shipped "pills" differentiate command vs. argument with an inline text background plus weight/colour (bold accent command, quieter accent argument). An editable RN TextInput can't pad or round an inline text background (RN #10807), and react-native-live-markdown can't either (same text rendering) — a true rounded chip needs a native rich-text widget or a WebView, both ruled out. We evaluated the react-native-live-markdown route, found it wouldn't deliver real chips, and stayed dependency-free. This is the platform ceiling and the final approach here, not a stopgap.
  • Editing a queued message flattens it to { text }. This is desktop's existing behaviour, not a mobile regression.

Known limitations (consciously accepted)

  • Typing a multi-line message flashes the conversation behind the composer. As the composer grows a line, the bottom inset changes and the virtualised conversation timeline re-pads and reflows — that reflow is the flash. We chased the native-side prop churn and the CSS (mask) causes and reverted all of it once they didn't pan out; the only genuine fix is restructuring how the native composer and the WebView-embedded conversation share space, which isn't worth it for a brief multi-line-typing transient.
  • Inline highlighting is a flat tint, not a chip (see decisions above).
  • Future spawn features (schema-form args, model/reasoning/speed, attachments) are not in this PR — but the spawnEntity / args plumbing is shaped so they're additive.

How to review

  • Start with Why native, not a WebView and Key decisions & trade-offs above for the design rationale and decision log.
  • Shared extraction: agents-runtime/src/composer-input.ts + the desktop repoints in agents-server-ui (import changes only — desktop behaviour is unchanged).
  • Mobile: NativeComposer.tsx, lib/slashAutocomplete.ts (pure + unit-tested), SessionScreen.tsx, NewSessionScreen.tsx, lib/agentsClient.ts.

Try it: open a Horton session, type / → a native menu of its commands appears; pick one → it inserts and highlights; send → Horton runs the command. Same flow on the new-session prompt.


Opened as a draft for early eyes — happy to walk through any of the decisions.

msfstef and others added 13 commits June 8, 2026 14:22
Plan to give the agents-mobile Horton prompt feature/experience parity
with the desktop ProseMirror composer (slash commands + structured
composer_input payloads) via a native RN composer rather than a WebView,
reusing the platform-agnostic runtime contract and slash-command grammar.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lift the slash-command serializer and grammar out of the DOM-only
ComposerEditor into agents-runtime/src/composer-input.ts so every composer
surface (desktop ProseMirror, upcoming mobile native TextInput) derives
nodes, highlighting, and the autocomplete trigger from one source of truth.

- Move serializeComposerInput, normalizeCommandName, and
  formatSlashCommandArgumentHint into composer-input.ts; export via /client.
- Extract both slash regexes as named constants: createSlashCommandTokenRegex
  (token) and SLASH_COMMAND_TRIGGER_REGEX (trigger), plus a reusable
  detectSlashCommandTrigger helper for trigger detection on plain strings.
- Repoint ComposerEditor's decoration/getSlashQuery regexes and the
  MessageInput/NewSessionView call sites at the shared exports.
- Move the serializer/hint tests into the runtime suite and add
  detectSlashCommandTrigger coverage; the dropped-token test pins the v1
  grammar boundary shared by desktop fallback and mobile.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Give the in-session Horton prompt feature parity with the desktop composer:
slash-command autocomplete and structured composer_input payloads, on a
native TextInput (no WebView), with a native keyboard-docked popover.

- NativeComposer.tsx: a useSlashAutocomplete hook (tracks caret via
  onSelectionChange, derives the trigger from the shared
  detectSlashCommandTrigger grammar, no caret coordinates) plus a native
  SlashCommandMenu docked above the input — all plain React state.
- slashAutocomplete.ts: pure, tested command filtering + a generic
  trigger->splice spine reusable by future node kinds (file/symbol/branch).
- SessionScreen: discover commands via the built-in db.collections.slashCommands
  live query, send via createSendComposerInputAction(serializeComposerInput(...)),
  render the menu above the existing keyboard-anchored composer card.

Scope notes (deliberate, per plan): in-session composer only — new-session
spawn parity needs a structured spawn payload (server work, out of scope);
no static fallback list (mobile entity-type schema lacks slash_commands);
editing still flattens to {text}, matching desktop's existing behavior
(no pills in v1, so nothing visibly flattens). Inline pills are deferred to
the gated live-markdown follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Apply the actionable findings from the critical review pass:

- Reconcile the plan: NewSessionScreen spawn parity is deferred (mobile
  spawnEntity takes a plain-string initialMessage with no message type), not
  "reuses the same component" — the body now agrees with the status note.
- Reset autocomplete caret state (slash.reset + pendingSelection) in
  startEditing/cancelEditing, matching the send path so all value-clearing
  paths are symmetric.
- Match desktop's SlashCommandMenu React key fallback (source:name, not name).
- Rename createMenuStyles -> createStyles, the single-styled-component
  convention used by every sibling mobile component.
- Move the NativeComposer import into the alphabetical components group.
- Trim verbose JSDoc that restated types (composer-input trigger helpers,
  NativeComposer rationale blocks, the insertion spine) down to the WHY.

No behavior change beyond the edit/cancel caret reset. Verdict from review:
sound, high in-session parity, no correctness bugs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d args

Forward-compatible client plumbing for new-session parity. No server change —
the spawn endpoint already accepts a structured initialMessage, an
initialMessageType, and an args record.

- initialMessage now accepts string | ComposerInputPayload; add
  initialMessageType (string passes through trimmed as before; an object is
  forwarded with its type).
- Generalize args from the hardcoded { workingDirectory } to a pass-through
  Record merged with workingDirectory — the same channel desktop uses for
  schema-form values and model settings, so future structured-arg / model /
  attachment work is additive, not a re-plumb.
- Add slash_commands to the entity-type schema (mirroring the desktop shape)
  so the new-session composer has an autocomplete source before any entity
  (and its live collection) exists.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring the new-session prompt to parity with the desktop NewSessionView: the
same native slash-command autocomplete as the in-session composer, and a
structured composer_input spawn.

- Reuse useSlashAutocomplete + SlashCommandMenu over the prompt TextInput,
  sourced from the selected type's static slash_commands (mapped to
  SlashCommandRow exactly as desktop does).
- Spawn with initialMessage = serializeComposerInput(text) and
  initialMessageType = composer_input when the prompt is non-empty; an empty
  prompt still spawns with no initial message, as before.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…omposer

Statically-declared slash commands are never materialised into the per-entity
slashCommands collection (createSlashCommandHelpers only writes rows on dynamic
register/unregister), so the in-session autocomplete was empty for agents whose
commands are all static — e.g. Horton. Fall back to the entity type's
slash_commands when the live collection is empty, mirroring the desktop
composer's effectiveSlashCommands (ChatView -> MessageInput). Also corrects the
stale comment that claimed static commands were materialised.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The trigger was derived from selection.start, which starts at 0 and is only
updated by onSelectionChange — RN delivers that a render after onChangeText (and
on some configs not at all for plain typing), so detectSlashCommandTrigger ran
against caret 0 and the menu never opened.

Default the caret to the end of the text (where a chat composer is almost always
edited) so the menu opens on the first `/`, and trust a reported caret only while
it matches the current value — which still enables mid-text triggers and
suppresses the menu during a range selection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Render recognized /command tokens (plus their declared argument words) as
styled "badges" inside the native composer, on both the in-session and
new-session inputs — the command in bold accent, its argument in a regular-
weight accent, sharing a faint tint so they read as one unit (mirroring the
desktop badge's command + argument slot).

- lib/slashAutocomplete.ts: computeHighlightRanges() derives the command and
  argument ranges from the shared serializer + the command's arg schema; pure
  and unit-tested.
- NativeComposer.renderComposerHighlights() renders the value as TextInput
  child <Text> spans (every segment coloured explicitly, since a nested colour
  is ignored when the TextInput sets its own).
- Switching the input from `value` to child <Text> breaks iOS's
  onContentSizeChange auto-grow (RN #13732), so size the input to content
  intrinsically within min/maxHeight instead, reporting height via the root
  onLayout. Net removal of the manual height-measurement machinery.

Note: a rounded/padded chip isn't achievable in an editable RN TextInput
(inline-text backgrounds can't be padded or rounded), so this is a flat tint —
the platform ceiling, matching how RN mention libraries style inline tokens.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Android's default soft-keyboard mode (`resize`) already lifts the
bottom-anchored composer card above the keyboard, and the manual
`translateY(-keyboardHeight)` then lifted it a second time, sending it toward
the top of the screen. Apply the translate on iOS only (which doesn't resize);
let Android's resize position the card.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@msfstef msfstef added the claude label Jun 8, 2026
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Electric Agents Desktop Builds

Build artifacts for commit fcf4538.

Platform Status Artifact
macOS Apple Silicon Passed DMG
macOS Intel Passed DMG
Windows x64 Passed Installer
Linux x64 Passed AppImage / deb

Workflow run

@netlify

netlify Bot commented Jun 8, 2026

Copy link
Copy Markdown

Deploy Preview for electric-next ready!

Name Link
🔨 Latest commit 8a054e0
🔍 Latest deploy log https://app.netlify.com/projects/electric-next/deploys/6a26ef4ffd25f60008618992
😎 Deploy Preview https://deploy-preview-4533--electric-next.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@codecov

codecov Bot commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.01980% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.45%. Comparing base (7892079) to head (fcf4538).
⚠️ Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
...agents-server-ui/src/components/ComposerEditor.tsx 0.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #4533       +/-   ##
===========================================
+ Coverage   32.48%   56.45%   +23.97%     
===========================================
  Files         216      359      +143     
  Lines       18368    39150    +20782     
  Branches     6478    11003     +4525     
===========================================
+ Hits         5967    22104    +16137     
- Misses      12369    16975     +4606     
- Partials       32       71       +39     
Flag Coverage Δ
packages/agents 70.83% <ø> (?)
packages/agents-mcp 77.54% <ø> (?)
packages/agents-mobile 71.42% <100.00%> (+4.50%) ⬆️
packages/agents-runtime 80.09% <100.00%> (?)
packages/agents-server 73.98% <ø> (+0.07%) ⬆️
packages/agents-server-ui 5.67% <0.00%> (-0.54%) ⬇️
packages/electric-ax 46.42% <ø> (?)
packages/experimental 87.73% <ø> (?)
packages/react-hooks 86.48% <ø> (?)
packages/start 82.83% <ø> (?)
packages/typescript-client 91.83% <ø> (?)
packages/y-electric 56.05% <ø> (?)
typescript 56.45% <98.01%> (+23.97%) ⬆️
unit-tests 56.45% <98.01%> (+23.97%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Electric Agents Mobile Build

Local mobile checks ran for commit fcf4538.

The EAS Android preview build was skipped because the mobile-eas-build label is not present.
Add the mobile-eas-build label to this PR to produce an installable preview build.

Workflow run

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@msfstef

msfstef commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author
Simulator Screenshot - iPhone 15 Pro - 2026-06-08 at 19 26 38 Simulator Screenshot - iPhone 15 Pro - 2026-06-08 at 19 27 13

@claude

claude Bot commented Jun 8, 2026

Copy link
Copy Markdown

Claude Code Review

Summary

Native (Expo/React Native) slash-command autocomplete + structured composer_input parity for the mobile Horton prompt, sharing the pure grammar/serializer via agents-runtime. The pure logic is cleanly factored and well-tested, and the RN platform quirks are documented thoroughly. Since iteration 2, the two new commits resolve the prior cosmetic/behavioral suggestions (badge over-extension and mid-text trigger) and remove the in-repo plan doc — all with new test coverage. This remains high-quality work; nothing outstanding beyond on-device verification.

What's Working Well

  • Mid-text trigger reworked into a pure, tested function. resolveSlashTrigger(value, selection) replaces the prior valueRef/reported.value === value freshness gate. The reasoning in the docstring is correct: RN updates value (onChangeText) a render before the matching onSelectionChange, so any "does this caret still match the value it was reported against" check is stale exactly on the render the value changes — which is what wedged the menu shut for mid-text edits. Bounds-checking the caret against the live value (0 <= start <= value.length, else fall back to end-of-text) is the right simplification, and it self-corrects because detectSlashCommandTrigger only fires when a /… actually precedes the caret. The hook is now just useState<Selection | null>useRef and the rangeSelected/fresh derivations are gone, with no dangling references.
  • Badge clamp is correct and minimal. computeHighlightRanges now clamps argument extension to nodes[n + 1]?.start ?? value.length. Since serializeComposerInput emits only slash_command nodes (no interleaved text nodes), nodes[n+1].start is precisely the next token boundary, so /deploy prod /plan highlights /deploy prod and /plan as two separate badges rather than swallowing /plan as an argument word. The for…offorEach((node, n) => conversion preserves the skip-unknown semantics.
  • Good new test coverage for both fixes. resolveSlashTrigger gets mid-text / null-selection / range-selection / out-of-bounds-stale / no-trigger cases, and computeHighlightRanges gets the explicit "never extends a badge into a following command token" case — all asserting exact ranges.
  • Carried-over strengths. The single-source-of-truth extraction into agents-runtime/src/composer-input.ts, the side-effect-free runtime serializer, and the forward-compatible spawnEntity widening remain clean (see prior iterations).

Issues Found

Critical (Must Fix)

None.

Important (Should Fix)

None.

Suggestions (Nice to Have)

1. (Carried) Children-controlled TextInput — confirm IME/Android on a real device. Unchanged: SessionScreen/NewSessionScreen drive content via nested <Text> children rather than value. Still the load-bearing change worth a physical-Android CJK/dictation pass (cursor-jump mid-composition, post-select selection splice not blurring). Not a code defect.

2. (Carried) Menu placement differs between the two surfaces. SessionScreen renders the menu above the input (keyboard-docked); NewSessionScreen renders it below in flow. Both reasonable; a deliberate consistency call is worth making.

Prior Suggestion (mid-text caret default) and the badge over-extension suggestion are now resolved — see below. Prior Suggestion #5 (plan doc at repo root) is resolved by fcf45383d removing COMPOSER_INPUT_MOBILE_PLAN.md.

Issue Conformance

No linked issue (draft for early feedback) — soft flag per convention. The PR description is thorough and now the canonical design log (the plan .md was intentionally removed as superseded). Implementation continues to match the "no server change" claim.

Previous Review Status

Incremental review (iteration 3). Changes since iteration 2 (98f47a0):

  • Resolved — prior Suggestion (mid-text caret). 8e303b3a0 extracts resolveSlashTrigger, fixing the mid-text trigger regression where the menu could stay shut; covered by new tests.
  • Resolved — prior Suggestion [Merged on #3] Write inserts, updates and deletes to Vaxine #2 (badge over-extension). Same commit clamps the badge at the next command token; covered by a new test.
  • Resolved — prior Suggestion Fix workflow location #5 (plan doc in repo). fcf45383d removes COMPOSER_INPUT_MOBILE_PLAN.md, deferring to the PR description.
  • Carried forward (unchanged): the on-device IME verification item and the menu-placement consistency note.

Review iteration: 3 | 2026-06-09

msfstef and others added 2 commits June 9, 2026 12:28
…ommand

The slash menu never opened for a command typed in the middle of existing
text (e.g. "hello /qui world"). The caret-freshness gate compared the
reported caret against a value that lagged one render, so it always fell
back to end-of-text — where the $-anchored grammar sees no trigger. Replace
it with resolveSlashTrigger, a pure helper that trusts the reported caret
whenever it indexes into the current value and only assumes end-of-text
until RN reports a caret. This stays at parity with the desktop composer,
which detects the trigger from the text before the cursor via the same
shared SLASH_COMMAND_TRIGGER_REGEX.

Also stop computeHighlightRanges from extending a command's argument badge
across a following /command token, so each recognized command keeps its own
badge (addresses the PR review's highlight-over-extension note).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…iption

The native mobile composer is implemented; its design rationale and decision
log now live in the PR description (which no longer references this file).
The deferred react-native-live-markdown "pills" route was evaluated and
dropped — it can't render real rounded chips either — so the text-background
+ font-weight badges are the final approach, not a stopgap awaiting a spike.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@msfstef msfstef marked this pull request as ready for review June 9, 2026 10:35
@msfstef msfstef requested review from kevin-dp and samwillis and removed request for samwillis June 9, 2026 11:57

@kevin-dp kevin-dp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work 👍

? q
.from({ slashCommand: db.collections.slashCommands as any })
.orderBy(({ slashCommand }: any) => slashCommand.name, `asc`)
: undefined,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this valid?
The docs type useLiveQuery's queryFn argument as (q) => QueryBuilder<TContext>. So it should return a QueryBuilder but this may return undefined.

[entityTypesCollection, entity?.type]
)
const slashCommands = useMemo<Array<SlashCommandRow>>(() => {
if (liveSlashCommands.length > 0) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if test isn't needed. We can remove it and just return (matchingTypes[0]?.slash_commands ?? []).map((command) => ({ ... }).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants